-- Logs begin at Fri 2022-08-19 15:33:03 UTC, end at Fri 2022-08-19 16:51:17 UTC. -- Aug 19 16:17:32.150905 dv-cinder-831247-23-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 19 16:17:32.158107 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 19 16:17:32.160112 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 19 16:17:32 2022] *** Aug 19 16:17:32.160112 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 19 16:17:32.160112 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 19 16:17:32.160112 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: nodename: dv-cinder-831247-23-iscsi Aug 19 16:17:32.160112 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: machine: x86_64 Aug 19 16:17:32.160112 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: clock source: unix Aug 19 16:17:32.160112 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: pcre jit disabled Aug 19 16:17:32.160112 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: detected number of CPU cores: 4 Aug 19 16:17:32.160610 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: current working directory: / Aug 19 16:17:32.160610 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: detected binary path: /usr/bin/uwsgi-core Aug 19 16:17:32.160610 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: your processes number limit is 47680 Aug 19 16:17:32.160610 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: your memory page size is 4096 bytes Aug 19 16:17:32.160610 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: detected max file descriptor number: 2048 Aug 19 16:17:32.160610 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: lock engine: pthread robust mutexes Aug 19 16:17:32.160610 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: thunder lock: enabled Aug 19 16:17:32.160610 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 19 16:17:32.160610 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 19 16:17:32.201366 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: Python main interpreter initialized at 0x560211d999c0 Aug 19 16:17:32.201366 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: python threads support enabled Aug 19 16:17:32.201366 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: your server socket listen backlog is limited to 100 connections Aug 19 16:17:32.201366 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: your mercy for graceful operations on workers is 90 seconds Aug 19 16:17:32.201739 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: mapped 403077 bytes (393 KB) for 2 cores Aug 19 16:17:32.201821 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: *** Operational MODE: preforking *** Aug 19 16:17:32.202090 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: *** uWSGI is running in multiple interpreter mode *** Aug 19 16:17:32.202090 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: spawned uWSGI master process (pid: 81538) Aug 19 16:17:32.202301 dv-cinder-831247-23-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 19 16:17:32.203379 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: spawned uWSGI worker 1 (pid: 81550, cores: 1) Aug 19 16:17:32.203379 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: spawned uWSGI worker 2 (pid: 81551, cores: 1) Aug 19 16:17:32.203379 dv-cinder-831247-23-iscsi devstack@keystone.service[81538]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 19 16:17:36.470836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 19 16:17:36.470836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: from cryptography.utils import int_from_bytes Aug 19 16:17:36.471621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 19 16:17:36.471621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: from cryptography.utils import int_from_bytes Aug 19 16:17:36.472162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 19 16:17:36.472162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: from cryptography.utils import int_from_bytes Aug 19 16:17:36.472612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 19 16:17:36.472612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: from cryptography.utils import int_from_bytes Aug 19 16:17:36.521316 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 19 16:17:36.521507 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 19 16:17:36.521698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] command line args: [] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 19 16:17:36.521792 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 19 16:17:36.521908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 19 16:17:36.522475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 19 16:17:36.522606 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] command line args: [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 19 16:17:36.522705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ================================================================================ {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 19 16:17:36.522872 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 19 16:17:36.522950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] admin_token = **** {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.523057 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ================================================================================ {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 19 16:17:36.523243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] config_dir = [] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.523352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.523438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] admin_token = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.523519 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] config_source = [] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.523626 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] debug = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.523721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] config_dir = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.523814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.523896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.524099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] config_source = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.524167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] debug = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.524349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.524445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.524575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.524671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.524824 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.524928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.525028 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.525135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.525242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] list_limit = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.525337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.525444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] log_config_append = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.525444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.525636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.525740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] log_dir = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.525835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_config_append = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.525934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] log_file = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.526034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.526034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_dir = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.526235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.526361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_file = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.526437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.526560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.526634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.526747 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.526826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.526953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.527035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.527035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.527170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.527170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.527361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.527439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.527510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.527631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.527883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.527976 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.528059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.528130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.528194 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.528268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.528342 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.528464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.528549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.528621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.528735 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.528862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.529046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.529149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.529367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.529549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.529708 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] publish_errors = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.529853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.529975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.530049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] publish_errors = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.530172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.530247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.530369 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.530466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.530545 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.530673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.530823 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.530953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.531029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] standard_threads = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.531143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.531216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.531329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] standard_threads = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.531414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.531531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.531609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.531750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] use_journal = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.531826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.531940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.532062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] use_json = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.532148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] use_journal = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.532219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] use_stderr = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.532288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] use_json = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.532362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] use_syslog = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.532431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] use_stderr = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.532506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.532576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] use_syslog = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.532706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.532804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 16:17:36.532971 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.533055 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.533232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.533310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.533473 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.533549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.533722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.533804 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.533960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.534038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.534193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.534273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.534431 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.534506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.534663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.534739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.534908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.534983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.535152 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.535228 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.535383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.535465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.535613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.535694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.535853 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.535935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.536085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.536167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.536327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.536409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.536561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.536637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.536815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.536908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.537069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.537151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.537317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.537392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.537537 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.538319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.538400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.538528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.538604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.538728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.538855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.539106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.539183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.539340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.539416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.539572 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.539650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.539809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.539885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.540039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.540119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.540271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.540349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.540505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.540582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.540739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.540839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.540994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.541072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.541230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.541311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.541462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.541538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.541999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.542090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.542090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.542228 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.542299 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.542373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.542373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.542507 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.542614 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.542691 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.542846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.542922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.543077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.543153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.543304 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.543384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.543533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.543607 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.543751 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.543823 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.543981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.544056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.544218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.544294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.544454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.544530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.544682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.544775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.544940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.545018 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.545317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.545442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.545539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.545638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.545733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.545827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.545929 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.546025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.546125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.546243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.546319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.546434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.546519 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.546591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.546750 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.546864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.546978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.547105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.547210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.547287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.547438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.547518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.547674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.547802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.547906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.547990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.548138 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.548214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.548385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.548510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.548655 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.548742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.548876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.548992 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.549077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.549150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.549315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.549403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.549550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.549671 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.549805 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.549928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.550038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.550123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.550275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.550350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.550512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.550639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.550744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.550829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.550978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.551055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.551212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.551288 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.551448 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.551535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.551681 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.551757 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.551915 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.551991 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.552149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.552236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.552381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.552463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.552616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.552692 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.552868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.552959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.553105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.553182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.553420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.553543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.553618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.553741 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.553857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.553983 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.554054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.554170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.554290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.554406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.554526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.554646 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.554761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.554880 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.555015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.555165 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.555240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.555335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.555412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.555563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.555648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.555794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.555881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.556023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.556107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.556257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.556335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.556487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.556571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.556728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.556851 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.556985 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.557062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.557231 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.557320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.557467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.557545 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.557721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.557803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] auth.external = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.557951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.558035 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.558204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.558280 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.558433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.external = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.558520 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.558663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.558751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] auth.password = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.558898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] auth.token = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.558984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.559139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.559219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.559377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.559458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.password = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.559612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.token = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.559700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.559847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.559924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.560080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.560156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.560324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.560410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.560555 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.560631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.560816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.560896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] credential.caching = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.561066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.561155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.561305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.561416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.561534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.561618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.561775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.561866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.562020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.562100 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.562255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.562333 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.562491 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.562578 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.562735 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.562815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.562969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.563048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.563219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.563307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.563470 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.563563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.563706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.563787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.563958 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.564048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.564215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.564294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.564469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.564553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.564704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] federation.caching = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.564826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.564966 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.565047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.565213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.565293 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.565446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.565534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.565678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.565757 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.565919 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.565997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.566154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.566235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.566381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.566459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.566624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.566704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.566858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.566939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.567102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.567181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.567334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.567412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.567576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.caching = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.567657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.567809 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.567894 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.568053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.568134 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.568287 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.568369 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.568533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.568614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.568784 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.568906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.569174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.569319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.569460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.569602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.569685 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.569746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.569815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.569965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.570055 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.570196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.570276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.570429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.570524 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.570675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.570756 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.570921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.571001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.571169 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.571259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.571425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.571506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.571672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.571825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.571945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.572032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.572170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.572251 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.572409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.572492 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.572642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.572741 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.572902 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.572984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.573134 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.573214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.573371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.573465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.573613 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.573694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.573852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.573932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.574100 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.574195 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.574334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.574414 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.574580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.574661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.574825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.574912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.575060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.575140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.575310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.575420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.575565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.575653 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.575819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.575900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.576063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.576142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.576332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.576438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.576569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.576694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.576834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.577032 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.577132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.577280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.577359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.577516 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.577594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.577760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.577847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.578005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.578082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.578237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.578315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.578467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.578553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.578709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.578788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.578952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.579029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.579190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.579276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.579426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.579506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.579659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.579736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.579889 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.579974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.580129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.580207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.580378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.580455 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.580608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.580693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.580871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.580951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.581109 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.581186 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.581352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.581442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.581588 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.581667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.581818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.581896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.582053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.582140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.582287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.582366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.582523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.582601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.582753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.582839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.582994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.583071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.583234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.583312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.583473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.583558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.583712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.583790 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.583946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.584030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.584173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.584259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.584419 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.585062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.585211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.585376 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.585524 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.585663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.585749 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.585826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.585923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.586004 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.586175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.586265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.586417 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.586498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.586657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.586737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.586897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.586986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.587135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.587217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.587385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.587464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.587617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.587705 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.587852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.587934 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.588087 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.588167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.588327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.588418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.588566 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.588652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.588824 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.588919 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.589060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.589151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.589298 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] resource.caching = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.589376 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.589544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.589620 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.589787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.589872 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.590020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.590097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.590261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.590339 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.590504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.590590 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.590739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.590816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.590983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.591061 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.591219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.591305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.591457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.591536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.591691 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] role.caching = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.591769 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.591936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.592015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] role.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.592163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.592240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.592412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.592489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.592656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.592737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.592918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] role.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.592997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.593164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.593241 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] role.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.593397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.593477 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.593641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.593718 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.593886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.593964 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.594132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.594212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.594366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.594444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.594611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.594689 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.594857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.594936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.595101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.595178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.595348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.595426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.595583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.595740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.595819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.595976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.596054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.596224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.596311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.596473 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.596552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.596718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.596953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.597032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.597194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.597274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.597447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.597525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.597698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.597784 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.597938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.598016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.598170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.598248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.598402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.598488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.598634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.598712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.598883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.598964 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.599131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.599219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.599368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.599447 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.599603 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.599679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.599836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.599920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.600068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.600175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.601005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] token.caching = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.601137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.601267 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.601404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.601528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.601653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.601778 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.601900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.602037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.602435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.602561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.602687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.602812 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.602939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.603072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.603205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.603333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.603459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.603585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.603710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.603836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.603970 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.604107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.604233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.604360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.604485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.604611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.604744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.604932 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.605056 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.605178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.605299 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.605422 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.605552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.605675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.605807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.605929 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.connection = **** {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.606061 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.606192 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.606315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.606446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.606569 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.606691 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.606812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.606933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.607055 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.607186 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.607315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.connection = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.607446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.607589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.607713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.607835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.607957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.608079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.608210 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.608332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.608455 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.608577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.608699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.608881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.609014 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.609142 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.609269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.609405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.609531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.609661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.609794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.609920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.610046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.610171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.610296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.610296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.610542 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.610668 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81550) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 19 16:17:36.610795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.610795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.610795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.610795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.610795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.611381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 16:17:36.611381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 19 16:17:36.638866 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.639097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.639251 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.639387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.639523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.639658 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.648591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.648771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.677351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.677583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.677808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.678045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.732025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.732633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.781196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.781428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.782312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.782549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.940107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.940470 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.940563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.940563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.940723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.940864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.940941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.941015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.941089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.941161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.941249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.941380 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.965478 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.965713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.965875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.965985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.966112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.966197 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.966337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.966421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.966554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.966638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.966763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.966846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.966972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.967181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.978350 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.978457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.991184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.991339 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.991425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.991565 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.991650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.991789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.991872 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.992007 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.992100 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.992219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.992302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.992431 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81550) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.992514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:36.992643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 16:17:37.009591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.009798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 19 16:17:37.010144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.010227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.010344 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 19 16:17:37.010444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 19 16:17:37.010623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.010707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.010890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.010974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 19 16:17:37.011154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.011239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.011422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.011661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.011743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.012172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.021018 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.021304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.021503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.021585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.021781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.021879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.022598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.022891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.029462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.029900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.032344 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.032633 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.032799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.032948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.033086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.033240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.033370 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.033530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.033652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.033797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.033934 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.034080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.034198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.034360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.034471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.034744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.034832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.035023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.035126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.035296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.035403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.035674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.058302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.058427 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.059757 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.059844 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.060043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.060137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.064692 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.064791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.065002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.065085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.065285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.065368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.072159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.072298 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.072445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.072587 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.076518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.076686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.076832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.077003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.077117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.077289 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.077400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.077575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.077679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.077856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.077945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.078136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.078216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.078409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.078488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.078680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.096098 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.096378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.096464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.096707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.100278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.100625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.103782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.104077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.104160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.104417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.108280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.108634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.111420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.111792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.114683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.114992 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.115150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.115279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.115364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.115563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.115656 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.115873 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.115992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.116158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.116270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.116583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.132668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.132972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.133058 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.133239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.133320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.133504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.133589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.133770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.140344 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.140643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.141911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.142190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.142272 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.142479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.144838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.145133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.145251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.145419 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.145505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.145705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.145796 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.145985 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.146069 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.146276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.158083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.158182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.158366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.158482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.158647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.158896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.158988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.159206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.159344 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.159490 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.159664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.159773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.159954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.160235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.175485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.175708 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.179212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.179318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.182140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.182227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.183596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.183692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.185093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.185182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.185377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.185462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.185672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.185756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.192145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.192233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.193632 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.193717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.197096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.197209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.197391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.197476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.197673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.197757 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.197963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.198054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.205924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.206010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.206219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.206302 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.206501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.206586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.213915 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.214199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.214283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81550) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 16:17:37.214478 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.214562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.214752 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.214836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.215022 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.215106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.215293 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.215379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.215565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.215648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.215839 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.215931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.216115 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.216197 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.216390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.216474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.216660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.216745 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.216943 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.217036 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.217212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.217297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.217477 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.217559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.217749 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81550) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 16:17:37.252115 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560211d999c0 pid: 81551 (default app) Aug 19 16:17:37.252608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560211d999c0 pid: 81550 (default app) Aug 19 16:17:37.255447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db96fef-a48b-4a09-b372-eedb93ff75bb None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:37.255734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db96fef-a48b-4a09-b372-eedb93ff75bb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:37.256006 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db96fef-a48b-4a09-b372-eedb93ff75bb None None] PATH_INFO: `/v3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:37.260262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1/1] 10.222.0.55 () {50 vars in 791 bytes} [Fri Aug 19 16:17:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:17:43.811133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38de39-4a60-4163-b46f-2631523ba22c None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:43.811644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38de39-4a60-4163-b46f-2631523ba22c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:43.811723 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38de39-4a60-4163-b46f-2631523ba22c None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:43.815910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1/2] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:17:43.821602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fba74a-5085-4f40-90bd-0e3ca37993d3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:43.821972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fba74a-5085-4f40-90bd-0e3ca37993d3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:43.822144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fba74a-5085-4f40-90bd-0e3ca37993d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:43.891397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: INFO dbcounter [None req-c0fba74a-5085-4f40-90bd-0e3ca37993d3 None None] Registered counter for database keystone Aug 19 16:17:43.910581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG oslo_db.sqlalchemy.engines [None req-c0fba74a-5085-4f40-90bd-0e3ca37993d3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81551) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 19 16:17:43.916928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writer thread running {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 19 16:17:44.097624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG passlib.handlers.bcrypt [None req-c0fba74a-5085-4f40-90bd-0e3ca37993d3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81551) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 19 16:17:44.098047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG passlib.handlers.bcrypt [None req-c0fba74a-5085-4f40-90bd-0e3ca37993d3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81551) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 19 16:17:44.155813 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c0fba74a-5085-4f40-90bd-0e3ca37993d3 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:44.157451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c0fba74a-5085-4f40-90bd-0e3ca37993d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:44.284390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 2/3] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 19 16:17:44.748817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38de39-4a60-4163-b46f-2631523ba22c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:44.817321 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: INFO dbcounter [None req-cc38de39-4a60-4163-b46f-2631523ba22c None None] Registered counter for database keystone Aug 19 16:17:44.839246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG oslo_db.sqlalchemy.engines [None req-cc38de39-4a60-4163-b46f-2631523ba22c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81550) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 19 16:17:44.845397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writer thread running {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 19 16:17:44.968108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38de39-4a60-4163-b46f-2631523ba22c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:44.972362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef75f1-f13d-42e1-ab43-51632a8b85ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:44.973023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef75f1-f13d-42e1-ab43-51632a8b85ed admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:44.973307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef75f1-f13d-42e1-ab43-51632a8b85ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:44.973580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef75f1-f13d-42e1-ab43-51632a8b85ed admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:44.982601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef75f1-f13d-42e1-ab43-51632a8b85ed admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:45.034404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: WARNING keystone.common.rbac_enforcer.enforcer [None req-44ef75f1-f13d-42e1-ab43-51632a8b85ed admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 19 16:17:45.034749 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef75f1-f13d-42e1-ab43-51632a8b85ed admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:45.045025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 2/4] 10.222.0.55 () {58 vars in 1324 bytes} [Fri Aug 19 16:17:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:17:45.050940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fba74a-5085-4f40-90bd-0e3ca37993d3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:45.065843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fba74a-5085-4f40-90bd-0e3ca37993d3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:45.070860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f8d19-71ad-4531-89c9-37da882b1190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:45.071973 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f8d19-71ad-4531-89c9-37da882b1190 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:45.072259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f8d19-71ad-4531-89c9-37da882b1190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:45.072534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f8d19-71ad-4531-89c9-37da882b1190 admin admin] PATH_INFO: `/v3/projects/61d450c451d94f27b564ec08cc487161` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:45.074123 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f8d19-71ad-4531-89c9-37da882b1190 admin admin] RBAC: Authorizing `identity:get_project(project_id=61d450c451d94f27b564ec08cc487161)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:45.127710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: WARNING keystone.common.rbac_enforcer.enforcer [None req-687f8d19-71ad-4531-89c9-37da882b1190 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 19 16:17:45.128059 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f8d19-71ad-4531-89c9-37da882b1190 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:45.133291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 3/5] 10.222.0.55 () {56 vars in 1207 bytes} [Fri Aug 19 16:17:45 2022] GET /identity/v3/projects/61d450c451d94f27b564ec08cc487161 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:17:47.024829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c06d6b-522b-4f86-8dca-9b2a9a4cab19 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:47.025370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c06d6b-522b-4f86-8dca-9b2a9a4cab19 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:47.025370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c06d6b-522b-4f86-8dca-9b2a9a4cab19 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:47.029514 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 3/6] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:17:47.034903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a2e8c-4478-405b-bb7a-4fdb6a8f6b76 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:47.035286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a2e8c-4478-405b-bb7a-4fdb6a8f6b76 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:47.035463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a2e8c-4478-405b-bb7a-4fdb6a8f6b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:47.056007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ab1a2e8c-4478-405b-bb7a-4fdb6a8f6b76 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:47.057564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ab1a2e8c-4478-405b-bb7a-4fdb6a8f6b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:47.065335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 4/7] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 19 16:17:47.333779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c06d6b-522b-4f86-8dca-9b2a9a4cab19 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:47.346610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c06d6b-522b-4f86-8dca-9b2a9a4cab19 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:47.350911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbabe2-694c-42ca-b8f2-fc86a24e734d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:47.351500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbabe2-694c-42ca-b8f2-fc86a24e734d admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:47.351779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbabe2-694c-42ca-b8f2-fc86a24e734d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:47.352048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbabe2-694c-42ca-b8f2-fc86a24e734d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:47.359877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbabe2-694c-42ca-b8f2-fc86a24e734d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:47.361685 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbabe2-694c-42ca-b8f2-fc86a24e734d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:47.370345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 4/8] 10.222.0.55 () {58 vars in 1324 bytes} [Fri Aug 19 16:17:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:17:47.375759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a2e8c-4478-405b-bb7a-4fdb6a8f6b76 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:47.385521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a2e8c-4478-405b-bb7a-4fdb6a8f6b76 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:47.390001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d45af-3140-4036-90dd-cb5f1da4e894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:47.390629 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d45af-3140-4036-90dd-cb5f1da4e894 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:47.390913 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d45af-3140-4036-90dd-cb5f1da4e894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:47.391193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d45af-3140-4036-90dd-cb5f1da4e894 admin admin] PATH_INFO: `/v3/users/1decd3055af14f42987a890baebb81a7` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:47.392532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d45af-3140-4036-90dd-cb5f1da4e894 admin admin] RBAC: Authorizing `identity:get_user(user_id=1decd3055af14f42987a890baebb81a7)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:47.394395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d45af-3140-4036-90dd-cb5f1da4e894 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:47.395917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 5/9] 10.222.0.55 () {56 vars in 1198 bytes} [Fri Aug 19 16:17:47 2022] GET /identity/v3/users/1decd3055af14f42987a890baebb81a7 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:17:49.344356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4681-9af9-4813-8e5d-b12d6480c26a None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:49.344911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4681-9af9-4813-8e5d-b12d6480c26a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:49.344992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4681-9af9-4813-8e5d-b12d6480c26a None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:49.346258 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 5/10] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:17:49.351526 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790dfa15-e028-4202-9228-5fe00db4fb98 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:49.352108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790dfa15-e028-4202-9228-5fe00db4fb98 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:49.352108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790dfa15-e028-4202-9228-5fe00db4fb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:49.370728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-790dfa15-e028-4202-9228-5fe00db4fb98 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:49.371901 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-790dfa15-e028-4202-9228-5fe00db4fb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:49.389051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d01fb-438f-4de4-8ce7-1876c861f564 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:49.389335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d01fb-438f-4de4-8ce7-1876c861f564 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:49.389607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d01fb-438f-4de4-8ce7-1876c861f564 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:49.390909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 6/11] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:17:49.395906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bf4ea-f88f-4415-b267-5122a373f5b5 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:49.396188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bf4ea-f88f-4415-b267-5122a373f5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:49.396463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bf4ea-f88f-4415-b267-5122a373f5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:49.437680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG passlib.handlers.bcrypt [None req-588bf4ea-f88f-4415-b267-5122a373f5b5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81550) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 19 16:17:49.437865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG passlib.handlers.bcrypt [None req-588bf4ea-f88f-4415-b267-5122a373f5b5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81550) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 19 16:17:49.452635 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 6/12] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:17:49.483417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-588bf4ea-f88f-4415-b267-5122a373f5b5 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:49.485013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-588bf4ea-f88f-4415-b267-5122a373f5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:49.582609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 7/13] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:17:49.721737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790dfa15-e028-4202-9228-5fe00db4fb98 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:49.745557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:49.826590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790dfa15-e028-4202-9228-5fe00db4fb98 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:49.848702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bf4ea-f88f-4415-b267-5122a373f5b5 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:49.863209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb4e68-53fc-4c98-9474-1f8c6d404bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:49.863851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb4e68-53fc-4c98-9474-1f8c6d404bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:49.864126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb4e68-53fc-4c98-9474-1f8c6d404bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:49.864396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb4e68-53fc-4c98-9474-1f8c6d404bf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:49.877186 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb4e68-53fc-4c98-9474-1f8c6d404bf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-57bb4e68-53fc-4c98-9474-1f8c6d404bf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:17:49.885827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:17:49.886864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:17:49.887702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 7/14] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:17:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:17:49.892510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb4e68-53fc-4c98-9474-1f8c6d404bf1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:49.952963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bf4ea-f88f-4415-b267-5122a373f5b5 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:49.955825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:49.973453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:49.992165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e6196-31c5-42d1-8c07-f1b287589485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:49.993066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e6196-31c5-42d1-8c07-f1b287589485 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:49.993351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e6196-31c5-42d1-8c07-f1b287589485 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:49.993453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb4e68-53fc-4c98-9474-1f8c6d404bf1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:49.993625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e6196-31c5-42d1-8c07-f1b287589485 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:50.026447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00cbac-be2a-4e12-ab25-6891bed5b0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:50.027051 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00cbac-be2a-4e12-ab25-6891bed5b0da None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:50.027330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00cbac-be2a-4e12-ab25-6891bed5b0da None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:50.027611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00cbac-be2a-4e12-ab25-6891bed5b0da None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:50.028283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00cbac-be2a-4e12-ab25-6891bed5b0da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:17:50.028583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00cbac-be2a-4e12-ab25-6891bed5b0da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:50.030016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00cbac-be2a-4e12-ab25-6891bed5b0da None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:50.041791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 8/15] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:17:50.047380 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00cbac-be2a-4e12-ab25-6891bed5b0da None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:50.059805 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e6196-31c5-42d1-8c07-f1b287589485 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:50.061367 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e6196-31c5-42d1-8c07-f1b287589485 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:50.145903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00cbac-be2a-4e12-ab25-6891bed5b0da None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:50.158694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 8/16] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 16:17:50.164456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e6196-31c5-42d1-8c07-f1b287589485 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:50.167618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:50.180440 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eeed3a-d323-4893-bff3-510299f30f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:50.181113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeed3a-d323-4893-bff3-510299f30f1d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:50.181399 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeed3a-d323-4893-bff3-510299f30f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:50.181675 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeed3a-d323-4893-bff3-510299f30f1d None admin] PATH_INFO: `/v3/users/1decd3055af14f42987a890baebb81a7` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:50.182827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eeed3a-d323-4893-bff3-510299f30f1d None admin] RBAC: Authorizing `identity:get_user(user_id=1decd3055af14f42987a890baebb81a7)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:50.184412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eeed3a-d323-4893-bff3-510299f30f1d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:50.186123 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 9/17] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:17:50 2022] GET /identity/v3/users/1decd3055af14f42987a890baebb81a7 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:17:50.191541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eeed3a-d323-4893-bff3-510299f30f1d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:50.206744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:50.263980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e6196-31c5-42d1-8c07-f1b287589485 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:50.293309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eeed3a-d323-4893-bff3-510299f30f1d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:50.296883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0094e63-fc5f-4781-8236-b276c7d52036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:50.297733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0094e63-fc5f-4781-8236-b276c7d52036 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:50.298011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0094e63-fc5f-4781-8236-b276c7d52036 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:50.298283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0094e63-fc5f-4781-8236-b276c7d52036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:50.317080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0094e63-fc5f-4781-8236-b276c7d52036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:50.318550 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0094e63-fc5f-4781-8236-b276c7d52036 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:50.327199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc73e8-2e11-4b16-8804-ec827f4255ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:50.327835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc73e8-2e11-4b16-8804-ec827f4255ac None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:50.328115 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc73e8-2e11-4b16-8804-ec827f4255ac None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:50.328390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc73e8-2e11-4b16-8804-ec827f4255ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:50.329546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc73e8-2e11-4b16-8804-ec827f4255ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:50.330995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc73e8-2e11-4b16-8804-ec827f4255ac None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:50.332541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 10/18] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:17:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-f0094e63-fc5f-4781-8236-b276c7d52036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:17:50.337637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:17:50.338457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:17:50.339246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:17:50.339246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:17:50.339246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:17:50.339396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc73e8-2e11-4b16-8804-ec827f4255ac None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:50.339466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 9/19] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:17:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:17:50.344285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0094e63-fc5f-4781-8236-b276c7d52036 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:50.403323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:50.419038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:50.439307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc73e8-2e11-4b16-8804-ec827f4255ac None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:50.448480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0094e63-fc5f-4781-8236-b276c7d52036 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:50.473462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294cf6c-24b8-4bf0-a2e4-77a5657c3181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:50.474010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294cf6c-24b8-4bf0-a2e4-77a5657c3181 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:50.474286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294cf6c-24b8-4bf0-a2e4-77a5657c3181 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:50.474559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294cf6c-24b8-4bf0-a2e4-77a5657c3181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:50.475282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294cf6c-24b8-4bf0-a2e4-77a5657c3181 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.domain.id=default, user.id=1decd3055af14f42987a890baebb81a7) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:17:50.476204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294cf6c-24b8-4bf0-a2e4-77a5657c3181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:50.477940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294cf6c-24b8-4bf0-a2e4-77a5657c3181 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:50.482361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5f60b-db1d-4f7f-a394-2824d16dd224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:50.482593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5f60b-db1d-4f7f-a394-2824d16dd224 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:50.482870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5f60b-db1d-4f7f-a394-2824d16dd224 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:50.483137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5f60b-db1d-4f7f-a394-2824d16dd224 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:50.483936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5f60b-db1d-4f7f-a394-2824d16dd224 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:17:50.484234 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5f60b-db1d-4f7f-a394-2824d16dd224 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:50.484795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 11/20] 10.222.0.55 () {56 vars in 1322 bytes} [Fri Aug 19 16:17:50 2022] GET /identity/v3/role_assignments?user.id=1decd3055af14f42987a890baebb81a7&scope.domain.id=default&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:17:50.485646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5f60b-db1d-4f7f-a394-2824d16dd224 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:50.508609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 10/21] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:17:52.700740 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403142e-7e85-4345-bf0a-f4bbebb5481c None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:52.701706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403142e-7e85-4345-bf0a-f4bbebb5481c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:52.702206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403142e-7e85-4345-bf0a-f4bbebb5481c None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:52.703717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 12/22] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:17:52.709055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e014e-fea1-4e38-8e28-def04e1858bc None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:52.710489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e014e-fea1-4e38-8e28-def04e1858bc None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:52.711022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e014e-fea1-4e38-8e28-def04e1858bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:52.730142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-6c5e014e-fea1-4e38-8e28-def04e1858bc None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:52.731667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6c5e014e-fea1-4e38-8e28-def04e1858bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:52.815991 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 11/23] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:17:52.865331 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0adf7-f75c-49bb-b653-f2d7d12d21ea None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:52.865988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0adf7-f75c-49bb-b653-f2d7d12d21ea None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:52.866666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0adf7-f75c-49bb-b653-f2d7d12d21ea None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:52.868197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 13/24] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:17:52.873351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850acbb5-d66e-4548-86af-6ded944a5e7c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:52.873877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850acbb5-d66e-4548-86af-6ded944a5e7c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:52.874352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850acbb5-d66e-4548-86af-6ded944a5e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:52.901842 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-850acbb5-d66e-4548-86af-6ded944a5e7c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:52.903392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-850acbb5-d66e-4548-86af-6ded944a5e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:52.955161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:52.983208 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 12/25] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:17:53.044018 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad6063-3166-49e9-be6e-d0f28c0e9bf3 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:53.044618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad6063-3166-49e9-be6e-d0f28c0e9bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:53.045136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad6063-3166-49e9-be6e-d0f28c0e9bf3 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:53.046650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 14/26] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:17:53.051642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94965702-2e59-46e6-9aa3-3421d2aeebee None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:53.052159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94965702-2e59-46e6-9aa3-3421d2aeebee None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:53.052626 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94965702-2e59-46e6-9aa3-3421d2aeebee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:53.077797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-94965702-2e59-46e6-9aa3-3421d2aeebee None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:53.079343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-94965702-2e59-46e6-9aa3-3421d2aeebee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:53.153568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 13/27] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:17:53.230633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9158b46-e2b3-43e1-a530-5f616b7c0a4a None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:53.231226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9158b46-e2b3-43e1-a530-5f616b7c0a4a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:53.231702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9158b46-e2b3-43e1-a530-5f616b7c0a4a None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:53.233529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 15/28] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:17:53.239054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a878a09-cb70-4e79-8921-6b31b702759c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:53.239567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a878a09-cb70-4e79-8921-6b31b702759c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:53.240091 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a878a09-cb70-4e79-8921-6b31b702759c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:53.269710 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-8a878a09-cb70-4e79-8921-6b31b702759c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:53.271230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8a878a09-cb70-4e79-8921-6b31b702759c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:53.280220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9158b46-e2b3-43e1-a530-5f616b7c0a4a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:53.409188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 14/29] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:17:53.410418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a878a09-cb70-4e79-8921-6b31b702759c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:53.446433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:53.486360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9158b46-e2b3-43e1-a530-5f616b7c0a4a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:53.547537 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a878a09-cb70-4e79-8921-6b31b702759c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:53.574416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f5391-6718-4466-8635-b5162a54d4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:53.575426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f5391-6718-4466-8635-b5162a54d4fa None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:53.575901 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f5391-6718-4466-8635-b5162a54d4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:53.576379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f5391-6718-4466-8635-b5162a54d4fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:53.594496 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5cfb0-8182-44dc-b9c1-9d45e33885cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:53.595148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5cfb0-8182-44dc-b9c1-9d45e33885cb None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:53.595406 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5cfb0-8182-44dc-b9c1-9d45e33885cb None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:53.595670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5cfb0-8182-44dc-b9c1-9d45e33885cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:53.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f5391-6718-4466-8635-b5162a54d4fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:53.599621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f5391-6718-4466-8635-b5162a54d4fa None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:53.605911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:53.625727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5cfb0-8182-44dc-b9c1-9d45e33885cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-948f5391-6718-4466-8635-b5162a54d4fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:17:53.630356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:17:53.631169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:17:53.631977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:17:53.631977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:17:53.631977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-5df5cfb0-8182-44dc-b9c1-9d45e33885cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:17:53.635619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:17:53.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 15/30] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:17:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:17:53.637780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5cfb0-8182-44dc-b9c1-9d45e33885cb None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:53.639112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 16/31] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:17:53.645127 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f5391-6718-4466-8635-b5162a54d4fa None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:53.759356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:53.764001 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5cfb0-8182-44dc-b9c1-9d45e33885cb None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:53.806097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a76b8-89eb-479e-bd2a-1f2a96a7a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:53.806942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a76b8-89eb-479e-bd2a-1f2a96a7a474 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:53.807415 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a76b8-89eb-479e-bd2a-1f2a96a7a474 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:53.807823 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a76b8-89eb-479e-bd2a-1f2a96a7a474 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:53.808637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a76b8-89eb-479e-bd2a-1f2a96a7a474 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:53.810213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a76b8-89eb-479e-bd2a-1f2a96a7a474 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:53.832564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f5391-6718-4466-8635-b5162a54d4fa None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:53.872205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27194683-b154-4480-a8b9-85fcdb9b519a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:53.872887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27194683-b154-4480-a8b9-85fcdb9b519a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:53.873174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27194683-b154-4480-a8b9-85fcdb9b519a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:53.873449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27194683-b154-4480-a8b9-85fcdb9b519a None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:53.874182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27194683-b154-4480-a8b9-85fcdb9b519a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:17:53.874543 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27194683-b154-4480-a8b9-85fcdb9b519a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:53.875935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27194683-b154-4480-a8b9-85fcdb9b519a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:53.885217 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 17/32] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:17:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:17:53.890983 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27194683-b154-4480-a8b9-85fcdb9b519a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:53.952263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 16/33] 10.222.0.55 () {60 vars in 1131 bytes} [Fri Aug 19 16:17:53 2022] POST /identity/v3/roles => generated 219 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:17:53.953691 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a76b8-89eb-479e-bd2a-1f2a96a7a474 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:53.955121 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:54.047031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27194683-b154-4480-a8b9-85fcdb9b519a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:54.101642 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fba343-4cf9-48d7-9eba-7793e9ec797c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:54.104606 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fba343-4cf9-48d7-9eba-7793e9ec797c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:54.106580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fba343-4cf9-48d7-9eba-7793e9ec797c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:54.107032 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fba343-4cf9-48d7-9eba-7793e9ec797c None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:54.107927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fba343-4cf9-48d7-9eba-7793e9ec797c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:17:54.110256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fba343-4cf9-48d7-9eba-7793e9ec797c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:54.111865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fba343-4cf9-48d7-9eba-7793e9ec797c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:54.145960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 18/34] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:17:54.146842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fba343-4cf9-48d7-9eba-7793e9ec797c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:54.160361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a76b8-89eb-479e-bd2a-1f2a96a7a474 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:54.199105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674afc80-30c9-40f6-afa1-7d444c39c1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:54.200036 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674afc80-30c9-40f6-afa1-7d444c39c1b4 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:54.200605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674afc80-30c9-40f6-afa1-7d444c39c1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:54.201046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674afc80-30c9-40f6-afa1-7d444c39c1b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:54.202107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674afc80-30c9-40f6-afa1-7d444c39c1b4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:54.203796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674afc80-30c9-40f6-afa1-7d444c39c1b4 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:54.226930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 17/35] 10.222.0.55 () {60 vars in 1131 bytes} [Fri Aug 19 16:17:53 2022] POST /identity/v3/roles => generated 225 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:17:54.228459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674afc80-30c9-40f6-afa1-7d444c39c1b4 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:54.260941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:54.282185 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fba343-4cf9-48d7-9eba-7793e9ec797c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:54.304611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:54.326929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e566936-6c23-4bca-916a-ab102094f853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:54.327595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e566936-6c23-4bca-916a-ab102094f853 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:54.327959 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e566936-6c23-4bca-916a-ab102094f853 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:54.328242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e566936-6c23-4bca-916a-ab102094f853 None admin] PATH_INFO: `/v3/users/1decd3055af14f42987a890baebb81a7` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:54.329563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e566936-6c23-4bca-916a-ab102094f853 None admin] RBAC: Authorizing `identity:get_user(user_id=1decd3055af14f42987a890baebb81a7)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:54.331292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e566936-6c23-4bca-916a-ab102094f853 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:54.332978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 19/36] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:17:54 2022] GET /identity/v3/users/1decd3055af14f42987a890baebb81a7 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:17:54.334416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1696a7-b0a1-43d9-be4c-d6a516b57fa0 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:54.334778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1696a7-b0a1-43d9-be4c-d6a516b57fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:54.335641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1696a7-b0a1-43d9-be4c-d6a516b57fa0 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:54.337298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 20/37] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:17:54.341964 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1696a7-b0a1-43d9-be4c-d6a516b57fa0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:54.370012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674afc80-30c9-40f6-afa1-7d444c39c1b4 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:54.414973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83680e-1e8a-4acc-9d1e-f5d3e85004db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:54.415868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83680e-1e8a-4acc-9d1e-f5d3e85004db None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:54.416331 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83680e-1e8a-4acc-9d1e-f5d3e85004db None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:54.416775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83680e-1e8a-4acc-9d1e-f5d3e85004db None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:54.417823 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83680e-1e8a-4acc-9d1e-f5d3e85004db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:54.419465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83680e-1e8a-4acc-9d1e-f5d3e85004db None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:54.457848 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 18/38] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:17:54 2022] POST /identity/v3/projects => generated 301 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:17:54.458498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26097163-07b6-48e2-b9b1-a2537bcc15fa None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:54.458834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26097163-07b6-48e2-b9b1-a2537bcc15fa None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:54.461742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26097163-07b6-48e2-b9b1-a2537bcc15fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:54.477666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1696a7-b0a1-43d9-be4c-d6a516b57fa0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:54.485740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-26097163-07b6-48e2-b9b1-a2537bcc15fa None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:54.488515 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-26097163-07b6-48e2-b9b1-a2537bcc15fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:54.529090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1dcef-e9af-489f-a984-1bdfc488bf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:54.530202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1dcef-e9af-489f-a984-1bdfc488bf77 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:54.530849 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1dcef-e9af-489f-a984-1bdfc488bf77 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:54.531412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1dcef-e9af-489f-a984-1bdfc488bf77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:54.533054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1dcef-e9af-489f-a984-1bdfc488bf77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:54.534945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1dcef-e9af-489f-a984-1bdfc488bf77 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:54.537034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 21/39] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:17:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:17:54.543449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1dcef-e9af-489f-a984-1bdfc488bf77 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:54.568043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:54.582203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 19/40] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:17:54.618753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb4387-58b4-474f-86dd-a3af42399b7e None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:54.619041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb4387-58b4-474f-86dd-a3af42399b7e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:54.619489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb4387-58b4-474f-86dd-a3af42399b7e None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:54.621813 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 20/41] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:17:54.626311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c86e8c-125b-438b-bc52-73dd64d95acd None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:54.626602 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c86e8c-125b-438b-bc52-73dd64d95acd None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:54.627188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c86e8c-125b-438b-bc52-73dd64d95acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:54.634128 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:54.648121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f5c86e8c-125b-438b-bc52-73dd64d95acd None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:54.649498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f5c86e8c-125b-438b-bc52-73dd64d95acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:54.697636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1dcef-e9af-489f-a984-1bdfc488bf77 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:54.738182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf7789-f188-4f84-a64c-139874f2ce42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:54.738855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf7789-f188-4f84-a64c-139874f2ce42 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:54.739138 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf7789-f188-4f84-a64c-139874f2ce42 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:54.739421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf7789-f188-4f84-a64c-139874f2ce42 None admin] PATH_INFO: `/v3/domains/default/users/1decd3055af14f42987a890baebb81a7/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:54.741922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf7789-f188-4f84-a64c-139874f2ce42 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1decd3055af14f42987a890baebb81a7, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:54.743353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf7789-f188-4f84-a64c-139874f2ce42 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:54.748789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 21/42] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:17:54.847519 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c86e8c-125b-438b-bc52-73dd64d95acd None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:54.860745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 22/43] 10.222.0.55 () {58 vars in 1361 bytes} [Fri Aug 19 16:17:54 2022] PUT /identity/v3/domains/default/users/1decd3055af14f42987a890baebb81a7/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:17:54.961356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c86e8c-125b-438b-bc52-73dd64d95acd None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:54.991844 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:55.003366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958eb9c-dfac-4c14-a49a-1b6d59d4709c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:55.003996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958eb9c-dfac-4c14-a49a-1b6d59d4709c None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:55.004278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958eb9c-dfac-4c14-a49a-1b6d59d4709c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:55.004551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958eb9c-dfac-4c14-a49a-1b6d59d4709c None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:55.005256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958eb9c-dfac-4c14-a49a-1b6d59d4709c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:55.006948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958eb9c-dfac-4c14-a49a-1b6d59d4709c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:55.020733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf7789-f188-4f84-a64c-139874f2ce42 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:55.021221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 22/44] 10.222.0.55 () {60 vars in 1131 bytes} [Fri Aug 19 16:17:54 2022] POST /identity/v3/roles => generated 223 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:17:55.139774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf7789-f188-4f84-a64c-139874f2ce42 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:55.154943 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:55.182322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccd152-1bdb-411a-b85c-0e558f304860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:55.182980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccd152-1bdb-411a-b85c-0e558f304860 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:55.183267 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccd152-1bdb-411a-b85c-0e558f304860 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:55.183546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccd152-1bdb-411a-b85c-0e558f304860 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:55.184842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ccd152-1bdb-411a-b85c-0e558f304860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:55.186640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ccd152-1bdb-411a-b85c-0e558f304860 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:55.191097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 23/45] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:17:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:17:55.197029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958eb9c-dfac-4c14-a49a-1b6d59d4709c None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:55.298496 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958eb9c-dfac-4c14-a49a-1b6d59d4709c None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:55.331344 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59c0d3-1823-4f13-8193-2f28d9bcff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:55.331960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59c0d3-1823-4f13-8193-2f28d9bcff12 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:55.332246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59c0d3-1823-4f13-8193-2f28d9bcff12 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:55.332527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59c0d3-1823-4f13-8193-2f28d9bcff12 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:55.333221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59c0d3-1823-4f13-8193-2f28d9bcff12 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:55.334682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59c0d3-1823-4f13-8193-2f28d9bcff12 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:55.363003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 23/46] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:17:55 2022] POST /identity/v3/projects => generated 312 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:17:56.793373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb90741-8684-4f1a-81be-488ff98c4fa3 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:56.793871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb90741-8684-4f1a-81be-488ff98c4fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:56.793952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb90741-8684-4f1a-81be-488ff98c4fa3 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:56.795276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 24/47] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:17:56.800344 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e50d00-4360-42e7-8216-b7b866047cad None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:56.800733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e50d00-4360-42e7-8216-b7b866047cad None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:56.800937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e50d00-4360-42e7-8216-b7b866047cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:56.819807 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-07e50d00-4360-42e7-8216-b7b866047cad None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:56.821032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-07e50d00-4360-42e7-8216-b7b866047cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:56.854519 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:56.888771 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 24/48] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:17:57.157560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb90741-8684-4f1a-81be-488ff98c4fa3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:57.250050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963bf77-668b-438f-a95f-82dfc07a9f44 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:57.250333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963bf77-668b-438f-a95f-82dfc07a9f44 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:57.250610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963bf77-668b-438f-a95f-82dfc07a9f44 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:57.251952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 25/49] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:17:57.256670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5c81a-b5bd-4faf-9392-83e641c1e797 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:57.256982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5c81a-b5bd-4faf-9392-83e641c1e797 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:57.257309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5c81a-b5bd-4faf-9392-83e641c1e797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:57.260744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb90741-8684-4f1a-81be-488ff98c4fa3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:57.275146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-38f5c81a-b5bd-4faf-9392-83e641c1e797 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:57.276346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-38f5c81a-b5bd-4faf-9392-83e641c1e797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:57.294520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5889ad-68a2-481d-aa6b-4a73f6b81a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:57.295156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5889ad-68a2-481d-aa6b-4a73f6b81a7e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:57.295443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5889ad-68a2-481d-aa6b-4a73f6b81a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:57.295723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5889ad-68a2-481d-aa6b-4a73f6b81a7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:57.306824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5889ad-68a2-481d-aa6b-4a73f6b81a7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-bf5889ad-68a2-481d-aa6b-4a73f6b81a7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:17:57.311633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:17:57.312578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:17:57.313358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 25/50] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:17:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:17:57.318505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5889ad-68a2-481d-aa6b-4a73f6b81a7e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:57.367573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:57.450142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 26/51] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:17:57.533464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5889ad-68a2-481d-aa6b-4a73f6b81a7e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:57.566885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0703ba3-74dd-4f08-abdf-7c2886b91e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:57.567489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0703ba3-74dd-4f08-abdf-7c2886b91e51 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:57.567771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0703ba3-74dd-4f08-abdf-7c2886b91e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:57.568046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0703ba3-74dd-4f08-abdf-7c2886b91e51 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:57.568704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0703ba3-74dd-4f08-abdf-7c2886b91e51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:17:57.569034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0703ba3-74dd-4f08-abdf-7c2886b91e51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:57.570517 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0703ba3-74dd-4f08-abdf-7c2886b91e51 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:57.578586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 26/52] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:17:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:17:57.583947 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5c81a-b5bd-4faf-9392-83e641c1e797 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:57.681212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:57.684258 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5c81a-b5bd-4faf-9392-83e641c1e797 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:57.716834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4eec90-e6e6-43e8-8f7b-8cf354ad26b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:57.717403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eec90-e6e6-43e8-8f7b-8cf354ad26b3 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:57.717682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eec90-e6e6-43e8-8f7b-8cf354ad26b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:57.717955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eec90-e6e6-43e8-8f7b-8cf354ad26b3 None admin] PATH_INFO: `/v3/users/1decd3055af14f42987a890baebb81a7` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:57.719306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4eec90-e6e6-43e8-8f7b-8cf354ad26b3 None admin] RBAC: Authorizing `identity:get_user(user_id=1decd3055af14f42987a890baebb81a7)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:57.720523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0703ba3-74dd-4f08-abdf-7c2886b91e51 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:57.720726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4eec90-e6e6-43e8-8f7b-8cf354ad26b3 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:57.722326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 27/53] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:17:57 2022] GET /identity/v3/users/1decd3055af14f42987a890baebb81a7 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:17:57.728472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4eec90-e6e6-43e8-8f7b-8cf354ad26b3 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:57.822577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0703ba3-74dd-4f08-abdf-7c2886b91e51 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:57.824979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:57.825923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4eec90-e6e6-43e8-8f7b-8cf354ad26b3 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:57.857123 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d9c60-5cf4-4c25-9eaf-7d7035e3c29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:57.857758 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d9c60-5cf4-4c25-9eaf-7d7035e3c29f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:57.858040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d9c60-5cf4-4c25-9eaf-7d7035e3c29f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:57.858378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d9c60-5cf4-4c25-9eaf-7d7035e3c29f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:57.859399 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d9c60-5cf4-4c25-9eaf-7d7035e3c29f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:57.859770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce453b2-0cfb-4e71-8a59-b4c8a9520951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:57.860397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce453b2-0cfb-4e71-8a59-b4c8a9520951 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:57.860679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce453b2-0cfb-4e71-8a59-b4c8a9520951 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:57.860865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d9c60-5cf4-4c25-9eaf-7d7035e3c29f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:57.860993 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce453b2-0cfb-4e71-8a59-b4c8a9520951 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:57.862108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce453b2-0cfb-4e71-8a59-b4c8a9520951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:57.862580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 27/54] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:17:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:17:57.863575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce453b2-0cfb-4e71-8a59-b4c8a9520951 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:57.867520 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 28/55] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:17:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:17:57.868110 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d9c60-5cf4-4c25-9eaf-7d7035e3c29f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:57.870793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce453b2-0cfb-4e71-8a59-b4c8a9520951 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:17:57.905897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:17:57.967400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d9c60-5cf4-4c25-9eaf-7d7035e3c29f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:57.970536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce453b2-0cfb-4e71-8a59-b4c8a9520951 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:17:58.000043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7758f10-dd77-49e4-bec9-150bb84fc128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:58.000651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7758f10-dd77-49e4-bec9-150bb84fc128 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:58.000961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7758f10-dd77-49e4-bec9-150bb84fc128 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:58.001239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7758f10-dd77-49e4-bec9-150bb84fc128 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:58.001892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7758f10-dd77-49e4-bec9-150bb84fc128 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:58.003064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017fb77-451b-43b4-a832-aa397ba8a266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:17:58.003349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7758f10-dd77-49e4-bec9-150bb84fc128 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:58.003663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017fb77-451b-43b4-a832-aa397ba8a266 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:58.003945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017fb77-451b-43b4-a832-aa397ba8a266 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:58.004218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017fb77-451b-43b4-a832-aa397ba8a266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:58.004970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017fb77-451b-43b4-a832-aa397ba8a266 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.domain.id=default, user.id=1decd3055af14f42987a890baebb81a7) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:17:58.005276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017fb77-451b-43b4-a832-aa397ba8a266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:17:58.007193 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017fb77-451b-43b4-a832-aa397ba8a266 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:17:58.017971 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 29/56] 10.222.0.55 () {56 vars in 1322 bytes} [Fri Aug 19 16:17:57 2022] GET /identity/v3/role_assignments?user.id=1decd3055af14f42987a890baebb81a7&scope.domain.id=default&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:17:58.033193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 28/57] 10.222.0.55 () {60 vars in 1140 bytes} [Fri Aug 19 16:17:57 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:17:59.934098 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6da7e-b3f0-481e-8811-c19a275c8a3b None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:59.934577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6da7e-b3f0-481e-8811-c19a275c8a3b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:59.934577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6da7e-b3f0-481e-8811-c19a275c8a3b None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:59.935902 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 30/58] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:17:59.940894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a84d7-0d05-4be8-a2fc-02614808ed58 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:17:59.941254 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a84d7-0d05-4be8-a2fc-02614808ed58 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:17:59.941448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a84d7-0d05-4be8-a2fc-02614808ed58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:17:59.959153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-609a84d7-0d05-4be8-a2fc-02614808ed58 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:17:59.961297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-609a84d7-0d05-4be8-a2fc-02614808ed58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:17:59.979016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:00.027533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 29/59] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:00.298677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6da7e-b3f0-481e-8811-c19a275c8a3b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:00.395693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:00.400575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6da7e-b3f0-481e-8811-c19a275c8a3b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:00.433637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa545897-f21f-47c9-979f-66e26715c5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:00.434254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa545897-f21f-47c9-979f-66e26715c5dc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:00.434534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa545897-f21f-47c9-979f-66e26715c5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:00.434805 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa545897-f21f-47c9-979f-66e26715c5dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:00.435906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa545897-f21f-47c9-979f-66e26715c5dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:00.437392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa545897-f21f-47c9-979f-66e26715c5dc None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:00.438933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 31/60] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:00.445090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a84d7-0d05-4be8-a2fc-02614808ed58 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:00.541550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a84d7-0d05-4be8-a2fc-02614808ed58 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:00.574995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fcc95-7294-45ba-8009-31899865fe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:00.575541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fcc95-7294-45ba-8009-31899865fe46 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:00.575825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fcc95-7294-45ba-8009-31899865fe46 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:00.576103 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fcc95-7294-45ba-8009-31899865fe46 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:00.576838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793fcc95-7294-45ba-8009-31899865fe46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:00.578235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793fcc95-7294-45ba-8009-31899865fe46 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:00.605005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 30/61] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:18:00 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:18:03.599390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157a1bd-271e-4fdf-9a6e-1ded2fd03488 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:03.599948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157a1bd-271e-4fdf-9a6e-1ded2fd03488 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:03.599948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157a1bd-271e-4fdf-9a6e-1ded2fd03488 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:03.602032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 32/62] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:03.607453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8316d73-8108-4ee1-8011-f41ce333c252 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:03.608252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8316d73-8108-4ee1-8011-f41ce333c252 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:03.608743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8316d73-8108-4ee1-8011-f41ce333c252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:03.616734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1296117-85b3-4ea3-9143-f2f3de192cd4 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:03.617272 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1296117-85b3-4ea3-9143-f2f3de192cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:03.617738 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1296117-85b3-4ea3-9143-f2f3de192cd4 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:03.619252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 33/63] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:03.637307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf3be7-1152-4123-afef-a7dee3d7e8cd None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:03.637909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf3be7-1152-4123-afef-a7dee3d7e8cd None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:03.638384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf3be7-1152-4123-afef-a7dee3d7e8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:03.639466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e8316d73-8108-4ee1-8011-f41ce333c252 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:03.640917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e8316d73-8108-4ee1-8011-f41ce333c252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:03.644300 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:03.675396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-8ebf3be7-1152-4123-afef-a7dee3d7e8cd None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:03.676599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8ebf3be7-1152-4123-afef-a7dee3d7e8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:03.755762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 31/64] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:03.804615 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 34/65] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 19 16:18:03.911089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbf4b4-884c-419c-9221-f18b6b58f7d7 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:03.911369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbf4b4-884c-419c-9221-f18b6b58f7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:03.911573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbf4b4-884c-419c-9221-f18b6b58f7d7 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:03.913059 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 32/66] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:03.929320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c3aa6-3352-4129-a734-2583069070fb None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:03.929607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c3aa6-3352-4129-a734-2583069070fb None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:03.929884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c3aa6-3352-4129-a734-2583069070fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:03.957784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-013c3aa6-3352-4129-a734-2583069070fb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:03.958986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-013c3aa6-3352-4129-a734-2583069070fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:04.047038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 35/67] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:04.061722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c8171-0313-48dc-b1d8-193af12e9ec3 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:04.062011 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c8171-0313-48dc-b1d8-193af12e9ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:04.062289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c8171-0313-48dc-b1d8-193af12e9ec3 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:04.063687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 33/68] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:04.068803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdd952-13d1-4bc7-a3d8-9088257efb19 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:04.069988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdd952-13d1-4bc7-a3d8-9088257efb19 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:04.070574 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdd952-13d1-4bc7-a3d8-9088257efb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:04.094661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-dcfdd952-13d1-4bc7-a3d8-9088257efb19 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:04.096155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-dcfdd952-13d1-4bc7-a3d8-9088257efb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:04.122494 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c8171-0313-48dc-b1d8-193af12e9ec3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:04.125557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:04.197361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 36/69] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:04.211207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdd952-13d1-4bc7-a3d8-9088257efb19 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:04.285919 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c8171-0313-48dc-b1d8-193af12e9ec3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:04.355693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2d494-9056-4ffa-b7dd-236f8dce1427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:04.356621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2d494-9056-4ffa-b7dd-236f8dce1427 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:04.356935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2d494-9056-4ffa-b7dd-236f8dce1427 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:04.357215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2d494-9056-4ffa-b7dd-236f8dce1427 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:04.368027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:04.371722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2d494-9056-4ffa-b7dd-236f8dce1427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-06e2d494-9056-4ffa-b7dd-236f8dce1427 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:04.377041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:04.377860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:04.378587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 34/70] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:04.379203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdd952-13d1-4bc7-a3d8-9088257efb19 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:04.381556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823b726-9923-4690-adfd-198789c2bc00 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:04.381556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823b726-9923-4690-adfd-198789c2bc00 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:04.381556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823b726-9923-4690-adfd-198789c2bc00 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:04.386750 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 35/71] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:04.386750 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823b726-9923-4690-adfd-198789c2bc00 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:04.417402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32aa608-89e4-4632-bbb7-e480fe360058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:04.418040 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32aa608-89e4-4632-bbb7-e480fe360058 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:04.418320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32aa608-89e4-4632-bbb7-e480fe360058 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:04.418595 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32aa608-89e4-4632-bbb7-e480fe360058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:04.429293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32aa608-89e4-4632-bbb7-e480fe360058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-d32aa608-89e4-4632-bbb7-e480fe360058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:04.434053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:04.434847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 37/72] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:18:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:18:04.440547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32aa608-89e4-4632-bbb7-e480fe360058 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:04.458477 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:04.499540 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823b726-9923-4690-adfd-198789c2bc00 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:04.534944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c4a75-f1f3-4786-bfb7-a71160108d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:04.535588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c4a75-f1f3-4786-bfb7-a71160108d52 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:04.535869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c4a75-f1f3-4786-bfb7-a71160108d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:04.536149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c4a75-f1f3-4786-bfb7-a71160108d52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:04.541822 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32aa608-89e4-4632-bbb7-e480fe360058 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:04.547228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c4a75-f1f3-4786-bfb7-a71160108d52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-479c4a75-f1f3-4786-bfb7-a71160108d52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:04.553118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:04.554555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:04.555489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 36/73] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:04.556274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520fa80-fe1f-45a7-a040-7fae6495acf1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:04.556274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520fa80-fe1f-45a7-a040-7fae6495acf1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:04.556519 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520fa80-fe1f-45a7-a040-7fae6495acf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:04.574699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e520fa80-fe1f-45a7-a040-7fae6495acf1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:04.575900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e520fa80-fe1f-45a7-a040-7fae6495acf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:04.579039 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f2987-3ff4-4b36-9800-f4997ec6b5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:04.579654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f2987-3ff4-4b36-9800-f4997ec6b5ab None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:04.579936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f2987-3ff4-4b36-9800-f4997ec6b5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:04.580210 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f2987-3ff4-4b36-9800-f4997ec6b5ab None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:04.580907 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f2987-3ff4-4b36-9800-f4997ec6b5ab None admin] RBAC: Adding query filter params (name=member) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:04.581211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f2987-3ff4-4b36-9800-f4997ec6b5ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:04.582594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f2987-3ff4-4b36-9800-f4997ec6b5ab None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:04.594230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 38/74] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:04.595025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f2987-3ff4-4b36-9800-f4997ec6b5ab None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:04.632653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:04.644509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 37/75] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:04.653771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520fa80-fe1f-45a7-a040-7fae6495acf1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:04.692211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:04.711207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f2987-3ff4-4b36-9800-f4997ec6b5ab None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:04.745856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66e37-f293-4661-9063-0f01311a7d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:04.746465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66e37-f293-4661-9063-0f01311a7d66 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:04.746741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66e37-f293-4661-9063-0f01311a7d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:04.747014 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66e37-f293-4661-9063-0f01311a7d66 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:04.747671 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f66e37-f293-4661-9063-0f01311a7d66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:04.747972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f66e37-f293-4661-9063-0f01311a7d66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:04.749373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f66e37-f293-4661-9063-0f01311a7d66 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:04.757468 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 39/76] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:04.758254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66e37-f293-4661-9063-0f01311a7d66 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:04.764229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520fa80-fe1f-45a7-a040-7fae6495acf1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:04.802253 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77535f-b802-4e16-ac76-48f303ece38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:04.803407 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77535f-b802-4e16-ac76-48f303ece38e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:04.804103 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77535f-b802-4e16-ac76-48f303ece38e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:04.804529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77535f-b802-4e16-ac76-48f303ece38e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:04.816790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77535f-b802-4e16-ac76-48f303ece38e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-7c77535f-b802-4e16-ac76-48f303ece38e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:04.822222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:04.823210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:04.824111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:04.824111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:04.824111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:04.824111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:04.824111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:04.824111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:04.824111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:04.824111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:04.824111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:04.824111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:04.824111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:04.824111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:04.825939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 38/77] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:04.826728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77535f-b802-4e16-ac76-48f303ece38e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:04.867504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66e37-f293-4661-9063-0f01311a7d66 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:04.882741 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:04.902962 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17986eb-c185-43ed-9168-f6ca3b32ead5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:04.903609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17986eb-c185-43ed-9168-f6ca3b32ead5 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:04.903894 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17986eb-c185-43ed-9168-f6ca3b32ead5 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:04.904173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17986eb-c185-43ed-9168-f6ca3b32ead5 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:04.904868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17986eb-c185-43ed-9168-f6ca3b32ead5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:04.905178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17986eb-c185-43ed-9168-f6ca3b32ead5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:04.906570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17986eb-c185-43ed-9168-f6ca3b32ead5 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:04.915001 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 40/78] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:04.915681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17986eb-c185-43ed-9168-f6ca3b32ead5 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:04.920126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:04.941845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77535f-b802-4e16-ac76-48f303ece38e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:04.975865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e2827-3776-4d04-97d6-4a9a9140b4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:04.976437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e2827-3776-4d04-97d6-4a9a9140b4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:04.976731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e2827-3776-4d04-97d6-4a9a9140b4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:04.977039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e2827-3776-4d04-97d6-4a9a9140b4b7 None admin] PATH_INFO: `/v3/users/17297aae8d6f4c7b801260350bd22b41` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:04.990542 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e2827-3776-4d04-97d6-4a9a9140b4b7 None admin] RBAC: Authorizing `identity:get_user(user_id=17297aae8d6f4c7b801260350bd22b41)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:04.992696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e2827-3776-4d04-97d6-4a9a9140b4b7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:04.994425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 39/79] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:04 2022] GET /identity/v3/users/17297aae8d6f4c7b801260350bd22b41 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:04.997977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e2827-3776-4d04-97d6-4a9a9140b4b7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:05.016089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17986eb-c185-43ed-9168-f6ca3b32ead5 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:05.049602 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36bdc2f-8c1b-4d76-9270-2b37261cb719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:05.050232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36bdc2f-8c1b-4d76-9270-2b37261cb719 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:05.050514 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36bdc2f-8c1b-4d76-9270-2b37261cb719 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:05.050788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36bdc2f-8c1b-4d76-9270-2b37261cb719 None admin] PATH_INFO: `/v3/users/1decd3055af14f42987a890baebb81a7` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:05.051913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36bdc2f-8c1b-4d76-9270-2b37261cb719 None admin] RBAC: Authorizing `identity:get_user(user_id=1decd3055af14f42987a890baebb81a7)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:05.053378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36bdc2f-8c1b-4d76-9270-2b37261cb719 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:05.055738 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 41/80] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:04 2022] GET /identity/v3/users/1decd3055af14f42987a890baebb81a7 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:05.060673 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36bdc2f-8c1b-4d76-9270-2b37261cb719 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:05.099726 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e2827-3776-4d04-97d6-4a9a9140b4b7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:05.109367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:05.126610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:05.135729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee8799-d4fb-4bcb-b00d-032230b78759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:05.136338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee8799-d4fb-4bcb-b00d-032230b78759 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:05.136618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee8799-d4fb-4bcb-b00d-032230b78759 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:05.136924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee8799-d4fb-4bcb-b00d-032230b78759 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:05.137589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee8799-d4fb-4bcb-b00d-032230b78759 None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:05.137889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee8799-d4fb-4bcb-b00d-032230b78759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:05.139264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee8799-d4fb-4bcb-b00d-032230b78759 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:05.147190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 40/81] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:05.148056 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee8799-d4fb-4bcb-b00d-032230b78759 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:05.156775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36bdc2f-8c1b-4d76-9270-2b37261cb719 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:05.189619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d70776-c549-4cd0-8c72-6734a8ade427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:05.190239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d70776-c549-4cd0-8c72-6734a8ade427 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:05.190518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d70776-c549-4cd0-8c72-6734a8ade427 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:05.190794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d70776-c549-4cd0-8c72-6734a8ade427 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:05.191882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d70776-c549-4cd0-8c72-6734a8ade427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:05.193370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d70776-c549-4cd0-8c72-6734a8ade427 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:05.195527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 42/82] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:05.196286 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d70776-c549-4cd0-8c72-6734a8ade427 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:05.252877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee8799-d4fb-4bcb-b00d-032230b78759 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:05.286707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359baeb-1a93-4350-b43e-305cc6529613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:05.287337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359baeb-1a93-4350-b43e-305cc6529613 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:05.287620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359baeb-1a93-4350-b43e-305cc6529613 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:05.287909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359baeb-1a93-4350-b43e-305cc6529613 None admin] PATH_INFO: `/v3/users/17297aae8d6f4c7b801260350bd22b41` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:05.289289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359baeb-1a93-4350-b43e-305cc6529613 None admin] RBAC: Authorizing `identity:get_user(user_id=17297aae8d6f4c7b801260350bd22b41)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:05.290746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359baeb-1a93-4350-b43e-305cc6529613 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:05.292144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d70776-c549-4cd0-8c72-6734a8ade427 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:05.292324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 41/83] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:05 2022] GET /identity/v3/users/17297aae8d6f4c7b801260350bd22b41 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:05.296666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359baeb-1a93-4350-b43e-305cc6529613 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:05.331025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef9c31-65c5-4b2d-b0e7-edd7098641e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:05.331632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef9c31-65c5-4b2d-b0e7-edd7098641e0 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:05.331920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef9c31-65c5-4b2d-b0e7-edd7098641e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:05.332200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef9c31-65c5-4b2d-b0e7-edd7098641e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:05.333547 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:05.343179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:05.394516 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359baeb-1a93-4350-b43e-305cc6529613 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:05.402173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef9c31-65c5-4b2d-b0e7-edd7098641e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:05.403689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef9c31-65c5-4b2d-b0e7-edd7098641e0 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:05.427284 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95982ece-5f0c-4e25-b17f-641912d466e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:05.427906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95982ece-5f0c-4e25-b17f-641912d466e4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:05.428192 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95982ece-5f0c-4e25-b17f-641912d466e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:05.428474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95982ece-5f0c-4e25-b17f-641912d466e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:05.495722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95982ece-5f0c-4e25-b17f-641912d466e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:05.497247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95982ece-5f0c-4e25-b17f-641912d466e4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:05.501243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 43/84] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:05.502080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef9c31-65c5-4b2d-b0e7-edd7098641e0 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:05.537205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:05.542547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:05.598138 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 42/85] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:05.599080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95982ece-5f0c-4e25-b17f-641912d466e4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:05.602364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef9c31-65c5-4b2d-b0e7-edd7098641e0 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:05.636764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b5ac3-2c6d-4548-a448-be13c508e4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:05.637420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b5ac3-2c6d-4548-a448-be13c508e4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:05.637705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b5ac3-2c6d-4548-a448-be13c508e4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:05.637982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b5ac3-2c6d-4548-a448-be13c508e4f9 None admin] PATH_INFO: `/v3/users/17297aae8d6f4c7b801260350bd22b41` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:05.639128 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b5ac3-2c6d-4548-a448-be13c508e4f9 None admin] RBAC: Authorizing `identity:get_user(user_id=17297aae8d6f4c7b801260350bd22b41)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:05.640606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b5ac3-2c6d-4548-a448-be13c508e4f9 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:05.642239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 44/86] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:05 2022] GET /identity/v3/users/17297aae8d6f4c7b801260350bd22b41 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:05.643023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b5ac3-2c6d-4548-a448-be13c508e4f9 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:05.700055 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95982ece-5f0c-4e25-b17f-641912d466e4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:05.733770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10256fc-1379-4698-9f95-2dc464c8e343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:05.734381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10256fc-1379-4698-9f95-2dc464c8e343 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:05.734670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10256fc-1379-4698-9f95-2dc464c8e343 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:05.734950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10256fc-1379-4698-9f95-2dc464c8e343 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:05.735590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10256fc-1379-4698-9f95-2dc464c8e343 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:05.737083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10256fc-1379-4698-9f95-2dc464c8e343 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:05.746952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b5ac3-2c6d-4548-a448-be13c508e4f9 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:05.747388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:05.757251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:05.759800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 43/87] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:18:05 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:18:05.759800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10256fc-1379-4698-9f95-2dc464c8e343 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:05.786188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41f1da-8da3-4acc-bda2-905049d6517e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:05.786843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41f1da-8da3-4acc-bda2-905049d6517e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:05.787084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41f1da-8da3-4acc-bda2-905049d6517e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:05.787404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41f1da-8da3-4acc-bda2-905049d6517e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:05.865855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41f1da-8da3-4acc-bda2-905049d6517e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:05.867945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41f1da-8da3-4acc-bda2-905049d6517e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:05.886987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10256fc-1379-4698-9f95-2dc464c8e343 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:05.933369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aea22-3887-4574-97fe-c909154d8c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:05.935528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aea22-3887-4574-97fe-c909154d8c82 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:05.935528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aea22-3887-4574-97fe-c909154d8c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:05.935528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aea22-3887-4574-97fe-c909154d8c82 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:05.935772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07aea22-3887-4574-97fe-c909154d8c82 None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:05.937233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07aea22-3887-4574-97fe-c909154d8c82 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:05.938888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 44/88] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:05 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:05.939666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aea22-3887-4574-97fe-c909154d8c82 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:05.981440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 45/89] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:05.982578 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41f1da-8da3-4acc-bda2-905049d6517e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:05.996144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:06.024812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:06.084408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aea22-3887-4574-97fe-c909154d8c82 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:06.114412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41f1da-8da3-4acc-bda2-905049d6517e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:06.135787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76357dda-8380-469f-bf6c-7aedd9742a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:06.136699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76357dda-8380-469f-bf6c-7aedd9742a5d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:06.137645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76357dda-8380-469f-bf6c-7aedd9742a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:06.137945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76357dda-8380-469f-bf6c-7aedd9742a5d None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:06.139484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76357dda-8380-469f-bf6c-7aedd9742a5d None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:06.141891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76357dda-8380-469f-bf6c-7aedd9742a5d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:06.144371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 45/90] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:05 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:06.145345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76357dda-8380-469f-bf6c-7aedd9742a5d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:06.156734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c56f19-dafa-4ca5-91aa-c629586cdbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:06.157374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c56f19-dafa-4ca5-91aa-c629586cdbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:06.157653 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c56f19-dafa-4ca5-91aa-c629586cdbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:06.157925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c56f19-dafa-4ca5-91aa-c629586cdbf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:06.232873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c56f19-dafa-4ca5-91aa-c629586cdbf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:06.234389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c56f19-dafa-4ca5-91aa-c629586cdbf3 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:06.246309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:06.288419 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76357dda-8380-469f-bf6c-7aedd9742a5d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:06.332433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:06.339543 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e474ef-2226-4913-829f-f510233ae12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:06.340218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e474ef-2226-4913-829f-f510233ae12f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:06.340603 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e474ef-2226-4913-829f-f510233ae12f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:06.340920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e474ef-2226-4913-829f-f510233ae12f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:06.341661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e474ef-2226-4913-829f-f510233ae12f None admin] RBAC: Adding query filter params (role.id=ac5a835695db44918d768db1bbb97589, scope.project.id=9164929bc082468aa5cb93ca3d057091, user.id=17297aae8d6f4c7b801260350bd22b41) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:06.341972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e474ef-2226-4913-829f-f510233ae12f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:06.343425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e474ef-2226-4913-829f-f510233ae12f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:06.351648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 46/91] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:06 2022] GET /identity/v3/role_assignments?user.id=17297aae8d6f4c7b801260350bd22b41&scope.project.id=9164929bc082468aa5cb93ca3d057091&role.id=ac5a835695db44918d768db1bbb97589 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:06.357824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e474ef-2226-4913-829f-f510233ae12f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:06.360889 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 46/92] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:06.361824 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c56f19-dafa-4ca5-91aa-c629586cdbf3 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:06.485944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c56f19-dafa-4ca5-91aa-c629586cdbf3 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:06.497136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:06.527769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb1a54-d1fc-4969-be34-1342265fa2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:06.528544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb1a54-d1fc-4969-be34-1342265fa2ea None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:06.528868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb1a54-d1fc-4969-be34-1342265fa2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:06.529154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb1a54-d1fc-4969-be34-1342265fa2ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:06.529905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb1a54-d1fc-4969-be34-1342265fa2ea None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=9164929bc082468aa5cb93ca3d057091, user.id=1decd3055af14f42987a890baebb81a7) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:06.530223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb1a54-d1fc-4969-be34-1342265fa2ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:06.531727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb1a54-d1fc-4969-be34-1342265fa2ea None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:06.539414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 47/93] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:06 2022] GET /identity/v3/role_assignments?user.id=1decd3055af14f42987a890baebb81a7&scope.project.id=9164929bc082468aa5cb93ca3d057091&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:06.540295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb1a54-d1fc-4969-be34-1342265fa2ea None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:06.622621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e474ef-2226-4913-829f-f510233ae12f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:06.682509 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb1a54-d1fc-4969-be34-1342265fa2ea None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:06.711183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c14c00-3f62-4509-b1b4-cfa2185972be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:06.711946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c14c00-3f62-4509-b1b4-cfa2185972be None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:06.712130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c14c00-3f62-4509-b1b4-cfa2185972be None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:06.712409 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c14c00-3f62-4509-b1b4-cfa2185972be None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:06.713650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c14c00-3f62-4509-b1b4-cfa2185972be None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:06.715132 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c14c00-3f62-4509-b1b4-cfa2185972be None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:06.726890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 47/94] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:06 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:06.733574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c14c00-3f62-4509-b1b4-cfa2185972be None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:06.741869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84658e67-34d3-4f76-9333-f8c6a6c7477e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:06.742866 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84658e67-34d3-4f76-9333-f8c6a6c7477e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:06.743157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84658e67-34d3-4f76-9333-f8c6a6c7477e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:06.743437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84658e67-34d3-4f76-9333-f8c6a6c7477e None admin] PATH_INFO: `/v3/projects/67530741fd554291a6a5507c304ee128` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:06.747214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84658e67-34d3-4f76-9333-f8c6a6c7477e None admin] RBAC: Authorizing `identity:get_project(project_id=67530741fd554291a6a5507c304ee128)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:06.748693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84658e67-34d3-4f76-9333-f8c6a6c7477e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:06.750427 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 48/95] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:06 2022] GET /identity/v3/projects/67530741fd554291a6a5507c304ee128 => generated 312 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:06.756547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84658e67-34d3-4f76-9333-f8c6a6c7477e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:06.856199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:06.902586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:06.929518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84658e67-34d3-4f76-9333-f8c6a6c7477e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:06.966322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b9e25-433c-4bbe-852d-e7a154027727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:06.966928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b9e25-433c-4bbe-852d-e7a154027727 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:06.967214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b9e25-433c-4bbe-852d-e7a154027727 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:06.967495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b9e25-433c-4bbe-852d-e7a154027727 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:06.968229 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b9e25-433c-4bbe-852d-e7a154027727 None admin] RBAC: Adding query filter params (role.id=ac5a835695db44918d768db1bbb97589, scope.project.id=67530741fd554291a6a5507c304ee128, user.id=17297aae8d6f4c7b801260350bd22b41) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:06.968569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b9e25-433c-4bbe-852d-e7a154027727 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:06.970268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b9e25-433c-4bbe-852d-e7a154027727 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:06.976552 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 49/96] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:06 2022] GET /identity/v3/role_assignments?user.id=17297aae8d6f4c7b801260350bd22b41&scope.project.id=67530741fd554291a6a5507c304ee128&role.id=ac5a835695db44918d768db1bbb97589 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:06.999261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c14c00-3f62-4509-b1b4-cfa2185972be None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:07.061980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943bffa-c043-48c1-a506-75f7cea054b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:07.062870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943bffa-c043-48c1-a506-75f7cea054b2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:07.063300 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943bffa-c043-48c1-a506-75f7cea054b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:07.063743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943bffa-c043-48c1-a506-75f7cea054b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:07.064640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943bffa-c043-48c1-a506-75f7cea054b2 None admin] RBAC: Adding query filter params (role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5, scope.project.id=9164929bc082468aa5cb93ca3d057091, user.id=17297aae8d6f4c7b801260350bd22b41) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:07.065133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943bffa-c043-48c1-a506-75f7cea054b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:07.066765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943bffa-c043-48c1-a506-75f7cea054b2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:07.077092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 48/97] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:06 2022] GET /identity/v3/role_assignments?user.id=17297aae8d6f4c7b801260350bd22b41&scope.project.id=9164929bc082468aa5cb93ca3d057091&role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 259 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:08.482951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cc26c-69d8-4b47-8b6a-69c4c627e158 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:08.483452 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cc26c-69d8-4b47-8b6a-69c4c627e158 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:08.483523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cc26c-69d8-4b47-8b6a-69c4c627e158 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:08.485388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 50/98] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:08.490144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb89b77-8f98-4481-9d46-0aaef7011416 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:08.491088 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb89b77-8f98-4481-9d46-0aaef7011416 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:08.491571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb89b77-8f98-4481-9d46-0aaef7011416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:08.515788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7eb89b77-8f98-4481-9d46-0aaef7011416 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:08.517356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7eb89b77-8f98-4481-9d46-0aaef7011416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:08.532249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dab522-f939-482d-a149-6d825c206169 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:08.532931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dab522-f939-482d-a149-6d825c206169 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:08.533416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dab522-f939-482d-a149-6d825c206169 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:08.534952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 51/99] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:18:08.541007 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a578d-6de7-4d0b-bd64-6e88c8885d69 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:08.541525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a578d-6de7-4d0b-bd64-6e88c8885d69 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:08.542029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a578d-6de7-4d0b-bd64-6e88c8885d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:08.573453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-806a578d-6de7-4d0b-bd64-6e88c8885d69 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:08.575077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-806a578d-6de7-4d0b-bd64-6e88c8885d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:08.652776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 49/100] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:08.679602 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 52/101] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:09.052571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb89b77-8f98-4481-9d46-0aaef7011416 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:09.133649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:09.197009 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb89b77-8f98-4481-9d46-0aaef7011416 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:09.235825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a578d-6de7-4d0b-bd64-6e88c8885d69 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:09.271649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c491fc-3e26-40dd-841f-cd94b876634d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:09.272737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c491fc-3e26-40dd-841f-cd94b876634d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:09.273319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c491fc-3e26-40dd-841f-cd94b876634d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:09.273811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c491fc-3e26-40dd-841f-cd94b876634d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:09.296065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:09.306199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c491fc-3e26-40dd-841f-cd94b876634d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-22c491fc-3e26-40dd-841f-cd94b876634d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:09.312699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:09.313522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:09.314306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 50/102] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:09.321450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c491fc-3e26-40dd-841f-cd94b876634d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:09.412042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a578d-6de7-4d0b-bd64-6e88c8885d69 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:09.455846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c7ee6-4cda-4379-842f-6b120f9b2e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:09.456577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c7ee6-4cda-4379-842f-6b120f9b2e97 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:09.456944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c7ee6-4cda-4379-842f-6b120f9b2e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:09.457267 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c7ee6-4cda-4379-842f-6b120f9b2e97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:09.458402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c7ee6-4cda-4379-842f-6b120f9b2e97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:09.460439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c7ee6-4cda-4379-842f-6b120f9b2e97 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:09.462080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 53/103] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:09.463288 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb960b-a15f-4edd-a73f-c62a33258fde None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:09.463583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb960b-a15f-4edd-a73f-c62a33258fde None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:09.463863 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb960b-a15f-4edd-a73f-c62a33258fde None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:09.465099 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 54/104] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:09.469677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb960b-a15f-4edd-a73f-c62a33258fde None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:09.495867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c491fc-3e26-40dd-841f-cd94b876634d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:09.528777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:09.565875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1b191-e453-4d6b-811a-18f9b887c67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:09.566927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1b191-e453-4d6b-811a-18f9b887c67b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:09.567516 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1b191-e453-4d6b-811a-18f9b887c67b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:09.567994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1b191-e453-4d6b-811a-18f9b887c67b None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:09.568899 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1b191-e453-4d6b-811a-18f9b887c67b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:09.569434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1b191-e453-4d6b-811a-18f9b887c67b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:09.571242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1b191-e453-4d6b-811a-18f9b887c67b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:09.577401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb960b-a15f-4edd-a73f-c62a33258fde None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:09.581048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:09.595832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 51/105] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:09.597343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96100c-73d4-4a5e-b8ba-42c786aedd72 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:09.597906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96100c-73d4-4a5e-b8ba-42c786aedd72 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:09.598325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96100c-73d4-4a5e-b8ba-42c786aedd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:09.625654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655c77e-4d54-42b0-a576-cc50449c1c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:09.626511 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655c77e-4d54-42b0-a576-cc50449c1c96 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:09.626935 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655c77e-4d54-42b0-a576-cc50449c1c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:09.627371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655c77e-4d54-42b0-a576-cc50449c1c96 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:09.628159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655c77e-4d54-42b0-a576-cc50449c1c96 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:09.629954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655c77e-4d54-42b0-a576-cc50449c1c96 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:09.645025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5e96100c-73d4-4a5e-b8ba-42c786aedd72 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:09.646527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5e96100c-73d4-4a5e-b8ba-42c786aedd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:09.692787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 55/106] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:18:09 2022] POST /identity/v3/projects => generated 302 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:18:09.692978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655c77e-4d54-42b0-a576-cc50449c1c96 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:09.798761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 52/107] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:09.800073 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b99106-e5ed-47b7-b290-317a279bcbe1 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:09.800360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b99106-e5ed-47b7-b290-317a279bcbe1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:09.800637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b99106-e5ed-47b7-b290-317a279bcbe1 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:09.802027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 53/108] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:09.803172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad38af1-ae7b-408c-a44c-eda8f449b06a None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:09.803460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad38af1-ae7b-408c-a44c-eda8f449b06a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:09.803795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad38af1-ae7b-408c-a44c-eda8f449b06a None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:09.805289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 54/109] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:09.810694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc2151-0fa4-4b22-b734-e993a2af1713 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:09.811159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc2151-0fa4-4b22-b734-e993a2af1713 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:09.811457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc2151-0fa4-4b22-b734-e993a2af1713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:09.845415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ebfc2151-0fa4-4b22-b734-e993a2af1713 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:09.846683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ebfc2151-0fa4-4b22-b734-e993a2af1713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:09.874606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655c77e-4d54-42b0-a576-cc50449c1c96 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:09.927609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30f760-f198-4b92-9348-26eb160ed9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:09.928250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30f760-f198-4b92-9348-26eb160ed9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:09.928595 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30f760-f198-4b92-9348-26eb160ed9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:09.929400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30f760-f198-4b92-9348-26eb160ed9a4 None admin] PATH_INFO: `/v3/users/ca9e4c836f1b4113ba4c42c043f594d0` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:09.932329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 55/110] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:09.935483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace2c14-aba8-4df5-bb5c-1d7d439fbd9f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:09.935483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace2c14-aba8-4df5-bb5c-1d7d439fbd9f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:09.935483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace2c14-aba8-4df5-bb5c-1d7d439fbd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:09.952001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4ace2c14-aba8-4df5-bb5c-1d7d439fbd9f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:09.953289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4ace2c14-aba8-4df5-bb5c-1d7d439fbd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:09.959559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30f760-f198-4b92-9348-26eb160ed9a4 None admin] RBAC: Authorizing `identity:get_user(user_id=ca9e4c836f1b4113ba4c42c043f594d0)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:09.961201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30f760-f198-4b92-9348-26eb160ed9a4 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:09.967410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 56/111] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:09 2022] GET /identity/v3/users/ca9e4c836f1b4113ba4c42c043f594d0 => generated 289 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:09.969385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30f760-f198-4b92-9348-26eb160ed9a4 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:09.997087 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:10.009385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:10.052375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 56/112] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:10.107500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a7d30-dbed-4b67-b859-c45807db894e None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:10.112783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a7d30-dbed-4b67-b859-c45807db894e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:10.112783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a7d30-dbed-4b67-b859-c45807db894e None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:10.112783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 57/113] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:18:10.117862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec370a85-c957-4f6c-b2f9-d574dff0c867 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:10.118290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec370a85-c957-4f6c-b2f9-d574dff0c867 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:10.118590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec370a85-c957-4f6c-b2f9-d574dff0c867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:10.146606 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ec370a85-c957-4f6c-b2f9-d574dff0c867 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:10.148363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ec370a85-c957-4f6c-b2f9-d574dff0c867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:10.156647 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30f760-f198-4b92-9348-26eb160ed9a4 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:10.219631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9ab7c-d3de-4e9d-8cd7-ad1bc2d1e64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:10.220246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9ab7c-d3de-4e9d-8cd7-ad1bc2d1e64f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:10.220526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9ab7c-d3de-4e9d-8cd7-ad1bc2d1e64f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:10.220815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9ab7c-d3de-4e9d-8cd7-ad1bc2d1e64f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:10.294066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 58/114] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:10.294944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec370a85-c957-4f6c-b2f9-d574dff0c867 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:10.316432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9ab7c-d3de-4e9d-8cd7-ad1bc2d1e64f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:10.318030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9ab7c-d3de-4e9d-8cd7-ad1bc2d1e64f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:10.339010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:10.449985 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:10.450278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 57/115] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 16:18:10.451117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9ab7c-d3de-4e9d-8cd7-ad1bc2d1e64f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:10.456649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec370a85-c957-4f6c-b2f9-d574dff0c867 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:10.500028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118350c5-af78-49cb-8f4a-bbee51000c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:10.500989 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118350c5-af78-49cb-8f4a-bbee51000c0b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:10.501272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118350c5-af78-49cb-8f4a-bbee51000c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:10.501548 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118350c5-af78-49cb-8f4a-bbee51000c0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:10.515257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118350c5-af78-49cb-8f4a-bbee51000c0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-118350c5-af78-49cb-8f4a-bbee51000c0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:10.521656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:10.522993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:10.523719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:10.523719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:10.523719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:10.523719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:10.523719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:10.523719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:10.523719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:10.523719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:10.523719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:10.523719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:10.523719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:10.523719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:10.524828 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 59/116] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:10.528797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118350c5-af78-49cb-8f4a-bbee51000c0b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:10.562412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9ab7c-d3de-4e9d-8cd7-ad1bc2d1e64f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:10.577456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:10.602119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2948cac-3579-4b2c-9d89-844edf031b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:10.602771 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2948cac-3579-4b2c-9d89-844edf031b5b None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:10.603306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2948cac-3579-4b2c-9d89-844edf031b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:10.603621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2948cac-3579-4b2c-9d89-844edf031b5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:10.615239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2948cac-3579-4b2c-9d89-844edf031b5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-f2948cac-3579-4b2c-9d89-844edf031b5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:10.620049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:10.620913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:10.621674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 58/117] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:10.622273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2948cac-3579-4b2c-9d89-844edf031b5b None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:10.657778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118350c5-af78-49cb-8f4a-bbee51000c0b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:10.692644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb9388-d56c-4039-80f0-921fe6fbdd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:10.693425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb9388-d56c-4039-80f0-921fe6fbdd72 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:10.693707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb9388-d56c-4039-80f0-921fe6fbdd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:10.693978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb9388-d56c-4039-80f0-921fe6fbdd72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:10.705595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb9388-d56c-4039-80f0-921fe6fbdd72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-26eb9388-d56c-4039-80f0-921fe6fbdd72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:10.710838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:10.711644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:10.712403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 60/118] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:18:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:10.713038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb9388-d56c-4039-80f0-921fe6fbdd72 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:10.733877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:10.745368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2948cac-3579-4b2c-9d89-844edf031b5b None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:10.787602 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df636733-83e6-4111-b51b-594963e2b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:10.788449 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df636733-83e6-4111-b51b-594963e2b93d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:10.788733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df636733-83e6-4111-b51b-594963e2b93d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:10.789034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df636733-83e6-4111-b51b-594963e2b93d None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:10.790163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df636733-83e6-4111-b51b-594963e2b93d None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:10.791622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df636733-83e6-4111-b51b-594963e2b93d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:10.798406 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 59/119] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:10 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:10.798406 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df636733-83e6-4111-b51b-594963e2b93d None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:10.826232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb9388-d56c-4039-80f0-921fe6fbdd72 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:10.841048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:10.862803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd8641-6d8a-4f59-84ba-d27770615c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:10.863415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd8641-6d8a-4f59-84ba-d27770615c8a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:10.863697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd8641-6d8a-4f59-84ba-d27770615c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:10.863974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd8641-6d8a-4f59-84ba-d27770615c8a None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:10.864642 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd8641-6d8a-4f59-84ba-d27770615c8a None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:10.864982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd8641-6d8a-4f59-84ba-d27770615c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:10.866407 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd8641-6d8a-4f59-84ba-d27770615c8a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:10.874677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 61/120] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:10.875309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd8641-6d8a-4f59-84ba-d27770615c8a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:10.906624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df636733-83e6-4111-b51b-594963e2b93d None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:10.942683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c0788-dd61-4a4b-8e54-31e0744953c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:10.943309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c0788-dd61-4a4b-8e54-31e0744953c1 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:10.943592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c0788-dd61-4a4b-8e54-31e0744953c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:10.943865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c0788-dd61-4a4b-8e54-31e0744953c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:10.956092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c0788-dd61-4a4b-8e54-31e0744953c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-3c6c0788-dd61-4a4b-8e54-31e0744953c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:10.960959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:10.961781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:10.962517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 60/121] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:10.963202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c0788-dd61-4a4b-8e54-31e0744953c1 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:10.969340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:10.990721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd8641-6d8a-4f59-84ba-d27770615c8a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:11.026822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347f635-4e76-40cc-a81e-4de2895e39c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:11.027445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347f635-4e76-40cc-a81e-4de2895e39c3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:11.027782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347f635-4e76-40cc-a81e-4de2895e39c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:11.028064 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347f635-4e76-40cc-a81e-4de2895e39c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:11.028739 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347f635-4e76-40cc-a81e-4de2895e39c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:11.029084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347f635-4e76-40cc-a81e-4de2895e39c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:11.030459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347f635-4e76-40cc-a81e-4de2895e39c3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:11.039816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 62/122] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:11.040666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347f635-4e76-40cc-a81e-4de2895e39c3 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:11.066840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:11.071510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c0788-dd61-4a4b-8e54-31e0744953c1 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:11.108313 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d7598-49c5-4066-bc21-cb609b43aca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:11.108956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d7598-49c5-4066-bc21-cb609b43aca5 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:11.109239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d7598-49c5-4066-bc21-cb609b43aca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:11.109522 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d7598-49c5-4066-bc21-cb609b43aca5 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:11.110188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d7598-49c5-4066-bc21-cb609b43aca5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:11.110486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d7598-49c5-4066-bc21-cb609b43aca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:11.111880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d7598-49c5-4066-bc21-cb609b43aca5 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:11.122377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 61/123] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:11.127447 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d7598-49c5-4066-bc21-cb609b43aca5 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:11.155404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347f635-4e76-40cc-a81e-4de2895e39c3 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:11.193460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af3378-6b1b-4a89-aabb-e408f77023ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:11.194963 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af3378-6b1b-4a89-aabb-e408f77023ed None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:11.195656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af3378-6b1b-4a89-aabb-e408f77023ed None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:11.195938 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af3378-6b1b-4a89-aabb-e408f77023ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:11.196677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af3378-6b1b-4a89-aabb-e408f77023ed None admin] RBAC: Adding query filter params (role.id=732934b017114352b11d54cb795862b1, scope.project.id=9164929bc082468aa5cb93ca3d057091, user.id=ca9e4c836f1b4113ba4c42c043f594d0) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:11.197021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af3378-6b1b-4a89-aabb-e408f77023ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:11.198511 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af3378-6b1b-4a89-aabb-e408f77023ed None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:11.216874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 63/124] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:11 2022] GET /identity/v3/role_assignments?user.id=ca9e4c836f1b4113ba4c42c043f594d0&scope.project.id=9164929bc082468aa5cb93ca3d057091&role.id=732934b017114352b11d54cb795862b1 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:11.216874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af3378-6b1b-4a89-aabb-e408f77023ed None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:11.229146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:11.248388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d7598-49c5-4066-bc21-cb609b43aca5 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:11.292628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3433b-6748-422d-892d-392cd786468e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:11.293311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3433b-6748-422d-892d-392cd786468e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:11.293596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3433b-6748-422d-892d-392cd786468e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:11.293877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3433b-6748-422d-892d-392cd786468e None admin] PATH_INFO: `/v3/users/17297aae8d6f4c7b801260350bd22b41` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:11.295080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3433b-6748-422d-892d-392cd786468e None admin] RBAC: Authorizing `identity:get_user(user_id=17297aae8d6f4c7b801260350bd22b41)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:11.296564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3433b-6748-422d-892d-392cd786468e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:11.298434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 62/125] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:11 2022] GET /identity/v3/users/17297aae8d6f4c7b801260350bd22b41 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:11.299065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3433b-6748-422d-892d-392cd786468e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:11.337318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:11.367794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af3378-6b1b-4a89-aabb-e408f77023ed None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:11.411956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1796ec-75e8-4655-b777-c9f71383ecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:11.413864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1796ec-75e8-4655-b777-c9f71383ecf3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:11.414435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1796ec-75e8-4655-b777-c9f71383ecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:11.414892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1796ec-75e8-4655-b777-c9f71383ecf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:11.415936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1796ec-75e8-4655-b777-c9f71383ecf3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:11.416704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1796ec-75e8-4655-b777-c9f71383ecf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:11.418546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1796ec-75e8-4655-b777-c9f71383ecf3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:11.419126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3433b-6748-422d-892d-392cd786468e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:11.430182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 64/126] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:11.431821 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1796ec-75e8-4655-b777-c9f71383ecf3 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:11.455224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f40cb1-f738-4294-8f57-4b955d701fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:11.455960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f40cb1-f738-4294-8f57-4b955d701fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:11.456418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f40cb1-f738-4294-8f57-4b955d701fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:11.456598 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f40cb1-f738-4294-8f57-4b955d701fe2 None admin] PATH_INFO: `/v3/users/17297aae8d6f4c7b801260350bd22b41` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:11.457860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f40cb1-f738-4294-8f57-4b955d701fe2 None admin] RBAC: Authorizing `identity:get_user(user_id=17297aae8d6f4c7b801260350bd22b41)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:11.459472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f40cb1-f738-4294-8f57-4b955d701fe2 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:11.461496 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 63/127] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:11 2022] GET /identity/v3/users/17297aae8d6f4c7b801260350bd22b41 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:11.462248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f40cb1-f738-4294-8f57-4b955d701fe2 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:11.513256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:11.551697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1796ec-75e8-4655-b777-c9f71383ecf3 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:11.566976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:11.570934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f40cb1-f738-4294-8f57-4b955d701fe2 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:11.593104 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c0e17-4792-4074-a1a3-26d44b8e7f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:11.594081 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c0e17-4792-4074-a1a3-26d44b8e7f41 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:11.594604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c0e17-4792-4074-a1a3-26d44b8e7f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:11.595103 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c0e17-4792-4074-a1a3-26d44b8e7f41 None admin] PATH_INFO: `/v3/users/1decd3055af14f42987a890baebb81a7` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:11.596670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c0e17-4792-4074-a1a3-26d44b8e7f41 None admin] RBAC: Authorizing `identity:get_user(user_id=1decd3055af14f42987a890baebb81a7)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:11.598518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c0e17-4792-4074-a1a3-26d44b8e7f41 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:11.605337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 65/128] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:11 2022] GET /identity/v3/users/1decd3055af14f42987a890baebb81a7 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:11.605337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c0e17-4792-4074-a1a3-26d44b8e7f41 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:11.615680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5797d4-ae09-4f1d-be5f-fc017a5d6d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:11.616980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5797d4-ae09-4f1d-be5f-fc017a5d6d0b None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:11.617278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5797d4-ae09-4f1d-be5f-fc017a5d6d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:11.617475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5797d4-ae09-4f1d-be5f-fc017a5d6d0b None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:11.621502 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5797d4-ae09-4f1d-be5f-fc017a5d6d0b None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:11.623841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5797d4-ae09-4f1d-be5f-fc017a5d6d0b None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:11.626674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 64/129] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:11 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:11.628142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5797d4-ae09-4f1d-be5f-fc017a5d6d0b None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:11.727560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c0e17-4792-4074-a1a3-26d44b8e7f41 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:11.765351 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109aea29-6784-4d91-b6f0-c6dad36c7bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:11.766046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aea29-6784-4d91-b6f0-c6dad36c7bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:11.766398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aea29-6784-4d91-b6f0-c6dad36c7bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:11.766747 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aea29-6784-4d91-b6f0-c6dad36c7bb6 None admin] PATH_INFO: `/v3/users/17297aae8d6f4c7b801260350bd22b41` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:11.767891 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5797d4-ae09-4f1d-be5f-fc017a5d6d0b None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:11.767998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109aea29-6784-4d91-b6f0-c6dad36c7bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=17297aae8d6f4c7b801260350bd22b41)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:11.769708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109aea29-6784-4d91-b6f0-c6dad36c7bb6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:11.771392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 66/130] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:11 2022] GET /identity/v3/users/17297aae8d6f4c7b801260350bd22b41 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:11.772861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109aea29-6784-4d91-b6f0-c6dad36c7bb6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:11.777558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:11.813392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95c168-e0f8-485e-85f0-2745d134b2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:11.814362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95c168-e0f8-485e-85f0-2745d134b2fc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:11.814841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95c168-e0f8-485e-85f0-2745d134b2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:11.815331 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95c168-e0f8-485e-85f0-2745d134b2fc None admin] PATH_INFO: `/v3/projects/67530741fd554291a6a5507c304ee128` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:11.816685 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95c168-e0f8-485e-85f0-2745d134b2fc None admin] RBAC: Authorizing `identity:get_project(project_id=67530741fd554291a6a5507c304ee128)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:11.818351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95c168-e0f8-485e-85f0-2745d134b2fc None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:11.820605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 65/131] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:11 2022] GET /identity/v3/projects/67530741fd554291a6a5507c304ee128 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:11.821642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95c168-e0f8-485e-85f0-2745d134b2fc None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:11.858296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:11.886803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109aea29-6784-4d91-b6f0-c6dad36c7bb6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:11.923770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2197e-0fef-44f7-854a-0c28bf59965f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:11.924457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2197e-0fef-44f7-854a-0c28bf59965f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:11.924685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2197e-0fef-44f7-854a-0c28bf59965f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:11.925096 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2197e-0fef-44f7-854a-0c28bf59965f None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:11.926291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2197e-0fef-44f7-854a-0c28bf59965f None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:11.927744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2197e-0fef-44f7-854a-0c28bf59965f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:11.934037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 67/132] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:11 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:11.934037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2197e-0fef-44f7-854a-0c28bf59965f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:11.936454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95c168-e0f8-485e-85f0-2745d134b2fc None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:11.972371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6f85d-2514-4a18-bc8f-98665bb7a4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:11.973442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6f85d-2514-4a18-bc8f-98665bb7a4f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:11.973927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6f85d-2514-4a18-bc8f-98665bb7a4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:11.974401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6f85d-2514-4a18-bc8f-98665bb7a4f3 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091/users/17297aae8d6f4c7b801260350bd22b41/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:11.977143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6f85d-2514-4a18-bc8f-98665bb7a4f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9164929bc082468aa5cb93ca3d057091, user_id=17297aae8d6f4c7b801260350bd22b41, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:11.979088 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6f85d-2514-4a18-bc8f-98665bb7a4f3 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:11.998151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 66/133] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:18:11 2022] PUT /identity/v3/projects/9164929bc082468aa5cb93ca3d057091/users/17297aae8d6f4c7b801260350bd22b41/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:11.999246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6f85d-2514-4a18-bc8f-98665bb7a4f3 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:12.013404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:12.049788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2197e-0fef-44f7-854a-0c28bf59965f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:12.090170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c5116-b2e5-43de-8155-0aa2596211a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:12.090867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c5116-b2e5-43de-8155-0aa2596211a7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:12.091150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c5116-b2e5-43de-8155-0aa2596211a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:12.091428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c5116-b2e5-43de-8155-0aa2596211a7 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:12.092736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c5116-b2e5-43de-8155-0aa2596211a7 None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:12.094283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c5116-b2e5-43de-8155-0aa2596211a7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:12.096343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 68/134] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:11 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:12.097227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c5116-b2e5-43de-8155-0aa2596211a7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:12.111773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:12.120089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6f85d-2514-4a18-bc8f-98665bb7a4f3 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:12.172081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a011d37-f9d9-479b-8fc7-124d5c6d9b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:12.173179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011d37-f9d9-479b-8fc7-124d5c6d9b9b None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:12.173699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011d37-f9d9-479b-8fc7-124d5c6d9b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:12.174168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011d37-f9d9-479b-8fc7-124d5c6d9b9b None admin] PATH_INFO: `/v3/projects/67530741fd554291a6a5507c304ee128/users/17297aae8d6f4c7b801260350bd22b41/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:12.176425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a011d37-f9d9-479b-8fc7-124d5c6d9b9b None admin] RBAC: Authorizing `identity:create_grant(project_id=67530741fd554291a6a5507c304ee128, user_id=17297aae8d6f4c7b801260350bd22b41, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:12.178122 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a011d37-f9d9-479b-8fc7-124d5c6d9b9b None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:12.193111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 67/135] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:18:11 2022] PUT /identity/v3/projects/67530741fd554291a6a5507c304ee128/users/17297aae8d6f4c7b801260350bd22b41/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:12.195671 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2377d28-374a-40cf-84c4-4b7e61b54b71 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:12.196159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2377d28-374a-40cf-84c4-4b7e61b54b71 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:12.196676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2377d28-374a-40cf-84c4-4b7e61b54b71 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:12.198256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 68/136] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:12.199256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2377d28-374a-40cf-84c4-4b7e61b54b71 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:12.237770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c5116-b2e5-43de-8155-0aa2596211a7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:12.284793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da56dc6-76cd-4f28-92ae-7e9a67709765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:12.284793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56dc6-76cd-4f28-92ae-7e9a67709765 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:12.284793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56dc6-76cd-4f28-92ae-7e9a67709765 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:12.284793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56dc6-76cd-4f28-92ae-7e9a67709765 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091/users/1decd3055af14f42987a890baebb81a7/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:12.285751 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da56dc6-76cd-4f28-92ae-7e9a67709765 None admin] RBAC: Authorizing `identity:create_grant(project_id=9164929bc082468aa5cb93ca3d057091, user_id=1decd3055af14f42987a890baebb81a7, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:12.288108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da56dc6-76cd-4f28-92ae-7e9a67709765 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:12.304809 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:12.308077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 69/137] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:18:12 2022] PUT /identity/v3/projects/9164929bc082468aa5cb93ca3d057091/users/1decd3055af14f42987a890baebb81a7/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:12.311284 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2c421-cee3-4fad-817f-faee8c71d966 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:12.311643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2c421-cee3-4fad-817f-faee8c71d966 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:12.311941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2c421-cee3-4fad-817f-faee8c71d966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:12.337188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2bd2c421-cee3-4fad-817f-faee8c71d966 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:12.338488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2bd2c421-cee3-4fad-817f-faee8c71d966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:12.424214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 70/138] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:12.445004 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2377d28-374a-40cf-84c4-4b7e61b54b71 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:12.482831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78616674-76a6-4019-8363-a2ecac915b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:12.483513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78616674-76a6-4019-8363-a2ecac915b43 None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:12.483795 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78616674-76a6-4019-8363-a2ecac915b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:12.484073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78616674-76a6-4019-8363-a2ecac915b43 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091/users/17297aae8d6f4c7b801260350bd22b41/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:12.486312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78616674-76a6-4019-8363-a2ecac915b43 None admin] RBAC: Authorizing `identity:create_grant(project_id=9164929bc082468aa5cb93ca3d057091, user_id=17297aae8d6f4c7b801260350bd22b41, role_id=f7bd4a6c906a4ad3b6d2f003f9c37bc5)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:12.487828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78616674-76a6-4019-8363-a2ecac915b43 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:12.514783 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 69/139] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:18:12 2022] PUT /identity/v3/projects/9164929bc082468aa5cb93ca3d057091/users/17297aae8d6f4c7b801260350bd22b41/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:12.729173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2c421-cee3-4fad-817f-faee8c71d966 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:12.856929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2c421-cee3-4fad-817f-faee8c71d966 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:12.889158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:12.895957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3afec2-5a23-4491-921f-986b6467d13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:12.897097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3afec2-5a23-4491-921f-986b6467d13d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:12.897417 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3afec2-5a23-4491-921f-986b6467d13d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:12.897713 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3afec2-5a23-4491-921f-986b6467d13d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:12.898856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3afec2-5a23-4491-921f-986b6467d13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:12.900329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3afec2-5a23-4491-921f-986b6467d13d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:12.902015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 71/140] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:12.907790 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78616674-76a6-4019-8363-a2ecac915b43 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:12.937831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:13.055479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78616674-76a6-4019-8363-a2ecac915b43 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:13.090852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c90f25-df88-4b45-aa7e-6df3e2b5ad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:13.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c90f25-df88-4b45-aa7e-6df3e2b5ad4b None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:13.091758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c90f25-df88-4b45-aa7e-6df3e2b5ad4b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:13.092038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c90f25-df88-4b45-aa7e-6df3e2b5ad4b None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:13.092700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c90f25-df88-4b45-aa7e-6df3e2b5ad4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:13.094198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c90f25-df88-4b45-aa7e-6df3e2b5ad4b None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:13.126897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 70/141] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:18:12 2022] POST /identity/v3/users => generated 283 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:18:13.579390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb79f0-346b-40ad-823d-1c2fe8cfc11b None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:13.579798 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb79f0-346b-40ad-823d-1c2fe8cfc11b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:13.579972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb79f0-346b-40ad-823d-1c2fe8cfc11b None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:13.581488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 72/142] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:13.597066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320368a-f6d4-4683-baa3-49bbb40ad52b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:13.597364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320368a-f6d4-4683-baa3-49bbb40ad52b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:13.597648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320368a-f6d4-4683-baa3-49bbb40ad52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:13.622139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-5320368a-f6d4-4683-baa3-49bbb40ad52b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:13.623357 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5320368a-f6d4-4683-baa3-49bbb40ad52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:13.696968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:13.705343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 71/143] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:14.264381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb79f0-346b-40ad-823d-1c2fe8cfc11b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:14.398920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb79f0-346b-40ad-823d-1c2fe8cfc11b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:14.439354 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372662e3-7f36-46bf-be44-816b8ca464cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:14.440008 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372662e3-7f36-46bf-be44-816b8ca464cb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:14.440290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372662e3-7f36-46bf-be44-816b8ca464cb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:14.440564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372662e3-7f36-46bf-be44-816b8ca464cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:14.453691 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372662e3-7f36-46bf-be44-816b8ca464cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-372662e3-7f36-46bf-be44-816b8ca464cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:14.458552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:14.459367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:14.460444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 73/144] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:14.465984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320368a-f6d4-4683-baa3-49bbb40ad52b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:14.505399 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504105b-8703-45d8-9df3-68152c7605bb None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:14.505706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504105b-8703-45d8-9df3-68152c7605bb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:14.505961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504105b-8703-45d8-9df3-68152c7605bb None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:14.507517 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 74/145] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:18:14.513013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39d353-2763-4d82-aee6-d7b46dfc1a17 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:14.513297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39d353-2763-4d82-aee6-d7b46dfc1a17 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:14.514140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39d353-2763-4d82-aee6-d7b46dfc1a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:14.531991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7b39d353-2763-4d82-aee6-d7b46dfc1a17 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:14.533181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7b39d353-2763-4d82-aee6-d7b46dfc1a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:14.596597 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320368a-f6d4-4683-baa3-49bbb40ad52b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:14.642816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:14.667650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 75/146] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:14.668838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee4b37-ac15-42fe-a238-912eb72f8ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:14.669569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee4b37-ac15-42fe-a238-912eb72f8ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:14.669929 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee4b37-ac15-42fe-a238-912eb72f8ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:14.670284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee4b37-ac15-42fe-a238-912eb72f8ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:14.671079 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee4b37-ac15-42fe-a238-912eb72f8ee4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:14.671454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee4b37-ac15-42fe-a238-912eb72f8ee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:14.672942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee4b37-ac15-42fe-a238-912eb72f8ee4 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:14.682749 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 72/147] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:14.688198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39d353-2763-4d82-aee6-d7b46dfc1a17 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:14.795807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39d353-2763-4d82-aee6-d7b46dfc1a17 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:14.831718 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80e760-c151-4e08-bc6d-bc235633000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:14.832435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80e760-c151-4e08-bc6d-bc235633000c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:14.832723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80e760-c151-4e08-bc6d-bc235633000c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:14.833038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80e760-c151-4e08-bc6d-bc235633000c None admin] PATH_INFO: `/v3/users/ca9e4c836f1b4113ba4c42c043f594d0` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:14.834204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80e760-c151-4e08-bc6d-bc235633000c None admin] RBAC: Authorizing `identity:get_user(user_id=ca9e4c836f1b4113ba4c42c043f594d0)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:14.836220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80e760-c151-4e08-bc6d-bc235633000c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:14.837920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 76/148] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:14 2022] GET /identity/v3/users/ca9e4c836f1b4113ba4c42c043f594d0 => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:14.843568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee4b37-ac15-42fe-a238-912eb72f8ee4 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:14.916019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:14.959285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee4b37-ac15-42fe-a238-912eb72f8ee4 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:15.013483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5bf1c-1c24-4657-b983-204dcfafa80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:15.014501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5bf1c-1c24-4657-b983-204dcfafa80e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:15.014976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5bf1c-1c24-4657-b983-204dcfafa80e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:15.015439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5bf1c-1c24-4657-b983-204dcfafa80e None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:15.016856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5bf1c-1c24-4657-b983-204dcfafa80e None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:15.018649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5bf1c-1c24-4657-b983-204dcfafa80e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:15.020530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 73/149] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:14 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:15.027191 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80e760-c151-4e08-bc6d-bc235633000c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:15.111577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:15.164721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80e760-c151-4e08-bc6d-bc235633000c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:15.202066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c71c8-bdc3-4209-83ad-8ee1db56a1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:15.202748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c71c8-bdc3-4209-83ad-8ee1db56a1f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:15.203028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c71c8-bdc3-4209-83ad-8ee1db56a1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:15.203308 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c71c8-bdc3-4209-83ad-8ee1db56a1f8 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091/users/ca9e4c836f1b4113ba4c42c043f594d0/roles/732934b017114352b11d54cb795862b1` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:15.205351 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c71c8-bdc3-4209-83ad-8ee1db56a1f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9164929bc082468aa5cb93ca3d057091, user_id=ca9e4c836f1b4113ba4c42c043f594d0, role_id=732934b017114352b11d54cb795862b1)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:15.206824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c71c8-bdc3-4209-83ad-8ee1db56a1f8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:15.234539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 77/150] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:18:15 2022] PUT /identity/v3/projects/9164929bc082468aa5cb93ca3d057091/users/ca9e4c836f1b4113ba4c42c043f594d0/roles/732934b017114352b11d54cb795862b1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:15.362876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d920b7a-414f-4513-8aa4-9f56caba9a69 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:15.363478 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d920b7a-414f-4513-8aa4-9f56caba9a69 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:15.363954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d920b7a-414f-4513-8aa4-9f56caba9a69 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:15.365582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 74/151] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:15.380252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672633f5-f547-41f8-8f68-f6e7d822fead None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:15.380799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672633f5-f547-41f8-8f68-f6e7d822fead None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:15.380874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672633f5-f547-41f8-8f68-f6e7d822fead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:15.408135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-672633f5-f547-41f8-8f68-f6e7d822fead None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:15.409782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-672633f5-f547-41f8-8f68-f6e7d822fead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:15.429304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d920b7a-414f-4513-8aa4-9f56caba9a69 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:15.500247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 78/152] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:15.586640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d920b7a-414f-4513-8aa4-9f56caba9a69 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:15.622024 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:15.635652 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff5b86-d44a-4ba6-879e-d0a46d83c3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:15.636301 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff5b86-d44a-4ba6-879e-d0a46d83c3af None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:15.636579 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff5b86-d44a-4ba6-879e-d0a46d83c3af None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:15.636881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff5b86-d44a-4ba6-879e-d0a46d83c3af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:15.655815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff5b86-d44a-4ba6-879e-d0a46d83c3af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-c8ff5b86-d44a-4ba6-879e-d0a46d83c3af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:15.660638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:15.661486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:15.662692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 75/153] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:18:15.669552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672633f5-f547-41f8-8f68-f6e7d822fead None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:15.686043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a713a-81ac-4d9f-b379-bb78cc3c9dd7 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:15.686351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a713a-81ac-4d9f-b379-bb78cc3c9dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:15.686605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a713a-81ac-4d9f-b379-bb78cc3c9dd7 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:15.688278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 76/154] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:15.695013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade7630-c54e-4da7-af4a-6bad4cf0b48c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:15.695299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade7630-c54e-4da7-af4a-6bad4cf0b48c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:15.695642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade7630-c54e-4da7-af4a-6bad4cf0b48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:15.721521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-dade7630-c54e-4da7-af4a-6bad4cf0b48c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:15.722728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-dade7630-c54e-4da7-af4a-6bad4cf0b48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:15.743397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:15.827690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 77/155] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:15.837370 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672633f5-f547-41f8-8f68-f6e7d822fead None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:15.891436 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee38fd-7572-4035-b50e-0d1500dd4e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:15.891982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee38fd-7572-4035-b50e-0d1500dd4e6f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:15.892265 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee38fd-7572-4035-b50e-0d1500dd4e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:15.892543 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee38fd-7572-4035-b50e-0d1500dd4e6f None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:15.893253 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee38fd-7572-4035-b50e-0d1500dd4e6f None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:15.893618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee38fd-7572-4035-b50e-0d1500dd4e6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:15.895021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee38fd-7572-4035-b50e-0d1500dd4e6f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:15.904573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 79/156] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:15.910190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade7630-c54e-4da7-af4a-6bad4cf0b48c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:16.049109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade7630-c54e-4da7-af4a-6bad4cf0b48c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:16.073943 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:16.094821 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee38fd-7572-4035-b50e-0d1500dd4e6f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:16.095488 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280be7e-9ea8-4a54-89e0-079b4acc8dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:16.096140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280be7e-9ea8-4a54-89e0-079b4acc8dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:16.096416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280be7e-9ea8-4a54-89e0-079b4acc8dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:16.096686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280be7e-9ea8-4a54-89e0-079b4acc8dd6 None admin] PATH_INFO: `/v3/users/17297aae8d6f4c7b801260350bd22b41` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:16.097973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280be7e-9ea8-4a54-89e0-079b4acc8dd6 None admin] RBAC: Authorizing `identity:get_user(user_id=17297aae8d6f4c7b801260350bd22b41)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:16.099434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280be7e-9ea8-4a54-89e0-079b4acc8dd6 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:16.101771 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 78/157] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:15 2022] GET /identity/v3/users/17297aae8d6f4c7b801260350bd22b41 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:16.107841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280be7e-9ea8-4a54-89e0-079b4acc8dd6 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:16.254233 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280be7e-9ea8-4a54-89e0-079b4acc8dd6 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:16.285170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee38fd-7572-4035-b50e-0d1500dd4e6f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:16.301113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:16.317238 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662d03f-f2d1-449b-a7f2-49600c35adc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:16.317857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662d03f-f2d1-449b-a7f2-49600c35adc9 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:16.318133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662d03f-f2d1-449b-a7f2-49600c35adc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:16.318402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662d03f-f2d1-449b-a7f2-49600c35adc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:16.345465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729bc4f-03cd-4ad7-a06e-be8afadab13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:16.346466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729bc4f-03cd-4ad7-a06e-be8afadab13e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:16.346901 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729bc4f-03cd-4ad7-a06e-be8afadab13e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:16.347332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729bc4f-03cd-4ad7-a06e-be8afadab13e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:16.366146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729bc4f-03cd-4ad7-a06e-be8afadab13e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-6729bc4f-03cd-4ad7-a06e-be8afadab13e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:16.377993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:16.379234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:16.380281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:16.380281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:16.380281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:16.380281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:16.380281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:16.380281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:16.380281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:16.380281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:16.380281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:16.380281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:16.380281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:16.380281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:16.383286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 80/158] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:16.389775 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66809eeb-bd0d-438c-b927-2bb4d1749a99 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:16.390262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66809eeb-bd0d-438c-b927-2bb4d1749a99 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:16.390693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66809eeb-bd0d-438c-b927-2bb4d1749a99 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:16.392168 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 81/159] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:16.397111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66809eeb-bd0d-438c-b927-2bb4d1749a99 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:16.408334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:16.441661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662d03f-f2d1-449b-a7f2-49600c35adc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:16.443408 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662d03f-f2d1-449b-a7f2-49600c35adc9 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:16.555962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66809eeb-bd0d-438c-b927-2bb4d1749a99 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:16.558594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 79/160] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 16:18:16.560098 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7a5a-e943-4870-9453-b95230dea4f5 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:16.560594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7a5a-e943-4870-9453-b95230dea4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:16.561069 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7a5a-e943-4870-9453-b95230dea4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:16.591978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3ecd7a5a-e943-4870-9453-b95230dea4f5 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:16.593283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89012a96-ca23-41e5-8d33-c4a867961751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:16.594013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3ecd7a5a-e943-4870-9453-b95230dea4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:16.594374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89012a96-ca23-41e5-8d33-c4a867961751 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:16.594845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89012a96-ca23-41e5-8d33-c4a867961751 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:16.595309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89012a96-ca23-41e5-8d33-c4a867961751 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:16.596159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89012a96-ca23-41e5-8d33-c4a867961751 None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:16.596650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89012a96-ca23-41e5-8d33-c4a867961751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:16.598235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89012a96-ca23-41e5-8d33-c4a867961751 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:16.608716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 82/161] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:16.609814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89012a96-ca23-41e5-8d33-c4a867961751 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:16.641290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:16.691524 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 80/162] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:16.692776 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd7a5a-e943-4870-9453-b95230dea4f5 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:16.717776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89012a96-ca23-41e5-8d33-c4a867961751 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:16.719975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:16.754748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff52faa-aa43-4ce8-9b57-ab2c14bc30ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:16.755708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff52faa-aa43-4ce8-9b57-ab2c14bc30ad None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:16.756188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff52faa-aa43-4ce8-9b57-ab2c14bc30ad None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:16.757196 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff52faa-aa43-4ce8-9b57-ab2c14bc30ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:16.769472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff52faa-aa43-4ce8-9b57-ab2c14bc30ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-fff52faa-aa43-4ce8-9b57-ab2c14bc30ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:16.776602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:16.777561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:16.778347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:16.778347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:16.778347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:16.778347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:16.778347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:16.778347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:16.778347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:16.778347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:16.778347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:16.778347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:16.778347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:16.778347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:16.781118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 83/163] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:16.782125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff52faa-aa43-4ce8-9b57-ab2c14bc30ad None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:16.896568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff52faa-aa43-4ce8-9b57-ab2c14bc30ad None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:16.905939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd7a5a-e943-4870-9453-b95230dea4f5 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:16.913049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:16.934850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffaab6f-8f9a-426f-b0a5-f350944b7b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:16.935489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaab6f-8f9a-426f-b0a5-f350944b7b47 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:16.936112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaab6f-8f9a-426f-b0a5-f350944b7b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:16.936676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaab6f-8f9a-426f-b0a5-f350944b7b47 None admin] PATH_INFO: `/v3/users/17297aae8d6f4c7b801260350bd22b41` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:16.938354 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffaab6f-8f9a-426f-b0a5-f350944b7b47 None admin] RBAC: Authorizing `identity:get_user(user_id=17297aae8d6f4c7b801260350bd22b41)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:16.939961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffaab6f-8f9a-426f-b0a5-f350944b7b47 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:16.941809 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 84/164] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:16 2022] GET /identity/v3/users/17297aae8d6f4c7b801260350bd22b41 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:16.942838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffaab6f-8f9a-426f-b0a5-f350944b7b47 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:16.966173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b521bf-7c16-43d8-a91b-e75e2e585681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:16.966819 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b521bf-7c16-43d8-a91b-e75e2e585681 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:16.967139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b521bf-7c16-43d8-a91b-e75e2e585681 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:16.967583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b521bf-7c16-43d8-a91b-e75e2e585681 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:16.968896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b521bf-7c16-43d8-a91b-e75e2e585681 None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:16.970494 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b521bf-7c16-43d8-a91b-e75e2e585681 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:16.972634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 81/165] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:16 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:16.973622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b521bf-7c16-43d8-a91b-e75e2e585681 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:17.088630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffaab6f-8f9a-426f-b0a5-f350944b7b47 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:17.113621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:17.138023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162f1c3-914b-4fb0-8923-37baff506833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:17.139407 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162f1c3-914b-4fb0-8923-37baff506833 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:17.140021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162f1c3-914b-4fb0-8923-37baff506833 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:17.142029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b521bf-7c16-43d8-a91b-e75e2e585681 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:17.142473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162f1c3-914b-4fb0-8923-37baff506833 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:17.143350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162f1c3-914b-4fb0-8923-37baff506833 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:17.145133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162f1c3-914b-4fb0-8923-37baff506833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:17.146751 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162f1c3-914b-4fb0-8923-37baff506833 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:17.163657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 85/166] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:17.165021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162f1c3-914b-4fb0-8923-37baff506833 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:17.205923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:17.217293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdfafa-e452-42cc-8b60-63ac3981e65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:17.218152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdfafa-e452-42cc-8b60-63ac3981e65e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:17.218644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdfafa-e452-42cc-8b60-63ac3981e65e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:17.219161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdfafa-e452-42cc-8b60-63ac3981e65e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:17.321920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdfafa-e452-42cc-8b60-63ac3981e65e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:17.323581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdfafa-e452-42cc-8b60-63ac3981e65e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:17.330582 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162f1c3-914b-4fb0-8923-37baff506833 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:17.399152 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51adc0-7734-44ae-b856-cc6555cdbf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:17.400430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51adc0-7734-44ae-b856-cc6555cdbf58 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:17.400899 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51adc0-7734-44ae-b856-cc6555cdbf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:17.401332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51adc0-7734-44ae-b856-cc6555cdbf58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:17.402232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51adc0-7734-44ae-b856-cc6555cdbf58 None admin] RBAC: Adding query filter params (role.id=ac5a835695db44918d768db1bbb97589, scope.project.id=9164929bc082468aa5cb93ca3d057091, user.id=17297aae8d6f4c7b801260350bd22b41) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:17.402694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51adc0-7734-44ae-b856-cc6555cdbf58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:17.404329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51adc0-7734-44ae-b856-cc6555cdbf58 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:17.419079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 86/167] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:17 2022] GET /identity/v3/role_assignments?user.id=17297aae8d6f4c7b801260350bd22b41&scope.project.id=9164929bc082468aa5cb93ca3d057091&role.id=ac5a835695db44918d768db1bbb97589 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:17.420150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51adc0-7734-44ae-b856-cc6555cdbf58 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:17.510057 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:17.537202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:17.542951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 82/168] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:17.544181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdfafa-e452-42cc-8b60-63ac3981e65e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:17.549356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51adc0-7734-44ae-b856-cc6555cdbf58 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:17.587379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e3bd8-26dd-4426-980c-5841ebbeb7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:17.588319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e3bd8-26dd-4426-980c-5841ebbeb7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:17.588772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e3bd8-26dd-4426-980c-5841ebbeb7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:17.589204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e3bd8-26dd-4426-980c-5841ebbeb7d2 None admin] PATH_INFO: `/v3/users/17297aae8d6f4c7b801260350bd22b41` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:17.590525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e3bd8-26dd-4426-980c-5841ebbeb7d2 None admin] RBAC: Authorizing `identity:get_user(user_id=17297aae8d6f4c7b801260350bd22b41)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:17.592155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e3bd8-26dd-4426-980c-5841ebbeb7d2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:17.594045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 87/169] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:17 2022] GET /identity/v3/users/17297aae8d6f4c7b801260350bd22b41 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:17.594973 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e3bd8-26dd-4426-980c-5841ebbeb7d2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:17.681093 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdfafa-e452-42cc-8b60-63ac3981e65e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:17.719980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760a524-1eb3-4d0e-8a34-eddd246d3fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:17.721073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760a524-1eb3-4d0e-8a34-eddd246d3fca None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:17.721559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760a524-1eb3-4d0e-8a34-eddd246d3fca None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:17.722030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760a524-1eb3-4d0e-8a34-eddd246d3fca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:17.734241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760a524-1eb3-4d0e-8a34-eddd246d3fca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-6760a524-1eb3-4d0e-8a34-eddd246d3fca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:17.739351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:17.740263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:17.742111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:17.742111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:17.742111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:17.742111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:17.742111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:17.742111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:17.742111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:17.742111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:17.742111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:17.742111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:17.742111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:17.742111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:17.744718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 83/170] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:18:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:17.745854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760a524-1eb3-4d0e-8a34-eddd246d3fca None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:17.793907 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e3bd8-26dd-4426-980c-5841ebbeb7d2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:17.801689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:17.863919 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760a524-1eb3-4d0e-8a34-eddd246d3fca None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:17.893514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab6306-394a-4516-a0f2-a2983be5c85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:17.894188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab6306-394a-4516-a0f2-a2983be5c85b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:17.894479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab6306-394a-4516-a0f2-a2983be5c85b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:17.894758 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab6306-394a-4516-a0f2-a2983be5c85b None admin] PATH_INFO: `/v3/projects/67530741fd554291a6a5507c304ee128` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:17.896213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab6306-394a-4516-a0f2-a2983be5c85b None admin] RBAC: Authorizing `identity:get_project(project_id=67530741fd554291a6a5507c304ee128)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:17.897504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab6306-394a-4516-a0f2-a2983be5c85b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:17.899194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 88/171] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:17 2022] GET /identity/v3/projects/67530741fd554291a6a5507c304ee128 => generated 312 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:17.899973 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab6306-394a-4516-a0f2-a2983be5c85b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:17.909176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcf545-fdd6-49fb-af61-87e5cacea2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:17.910179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcf545-fdd6-49fb-af61-87e5cacea2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:17.910649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcf545-fdd6-49fb-af61-87e5cacea2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:17.911121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcf545-fdd6-49fb-af61-87e5cacea2b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:17.956965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:17.985053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcf545-fdd6-49fb-af61-87e5cacea2b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:17.986909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcf545-fdd6-49fb-af61-87e5cacea2b3 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:18.035646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:18.096187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 84/172] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:18.097833 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f2121-0cde-4f50-8909-7b38d203a8b8 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.098352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f2121-0cde-4f50-8909-7b38d203a8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.098818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f2121-0cde-4f50-8909-7b38d203a8b8 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.100283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 85/173] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:18.101376 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab6306-394a-4516-a0f2-a2983be5c85b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:18.101986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f2121-0cde-4f50-8909-7b38d203a8b8 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:18.137104 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55528a-2611-461c-8661-bf173f0a2101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:18.137730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55528a-2611-461c-8661-bf173f0a2101 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.138016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55528a-2611-461c-8661-bf173f0a2101 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.138295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55528a-2611-461c-8661-bf173f0a2101 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.138968 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55528a-2611-461c-8661-bf173f0a2101 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:18.139288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55528a-2611-461c-8661-bf173f0a2101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:18.140780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55528a-2611-461c-8661-bf173f0a2101 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:18.149550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 89/174] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:18.150658 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c084f-f3aa-4f50-a0cd-d829e1cbc80d None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.150951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c084f-f3aa-4f50-a0cd-d829e1cbc80d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.151228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c084f-f3aa-4f50-a0cd-d829e1cbc80d None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.152437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 90/175] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:18.157478 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104533a-52b4-4e0b-9e2a-5e9f724d9c0e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.157781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104533a-52b4-4e0b-9e2a-5e9f724d9c0e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.158080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104533a-52b4-4e0b-9e2a-5e9f724d9c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.176926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2104533a-52b4-4e0b-9e2a-5e9f724d9c0e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:18.178054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2104533a-52b4-4e0b-9e2a-5e9f724d9c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:18.215938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f2121-0cde-4f50-8909-7b38d203a8b8 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:18.255074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a56f6-4bef-48d2-90b1-6683acb7631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:18.255694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a56f6-4bef-48d2-90b1-6683acb7631e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.255977 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a56f6-4bef-48d2-90b1-6683acb7631e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.256255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a56f6-4bef-48d2-90b1-6683acb7631e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.257480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a56f6-4bef-48d2-90b1-6683acb7631e None admin] RBAC: Adding query filter params (role.id=ac5a835695db44918d768db1bbb97589, scope.project.id=67530741fd554291a6a5507c304ee128, user.id=17297aae8d6f4c7b801260350bd22b41) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:18.257794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a56f6-4bef-48d2-90b1-6683acb7631e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:18.259260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a56f6-4bef-48d2-90b1-6683acb7631e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:18.260176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 91/176] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:18.261433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104533a-52b4-4e0b-9e2a-5e9f724d9c0e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:18.265766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 86/177] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:18 2022] GET /identity/v3/role_assignments?user.id=17297aae8d6f4c7b801260350bd22b41&scope.project.id=67530741fd554291a6a5507c304ee128&role.id=ac5a835695db44918d768db1bbb97589 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:18.272932 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a56f6-4bef-48d2-90b1-6683acb7631e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:18.294641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:18.305274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:18.419641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a56f6-4bef-48d2-90b1-6683acb7631e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:18.458835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104533a-52b4-4e0b-9e2a-5e9f724d9c0e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:18.497437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35751e29-58f8-47fa-8816-03ca429fb764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:18.498081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35751e29-58f8-47fa-8816-03ca429fb764 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.498359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35751e29-58f8-47fa-8816-03ca429fb764 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.498626 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35751e29-58f8-47fa-8816-03ca429fb764 None admin] PATH_INFO: `/v3/users/1decd3055af14f42987a890baebb81a7` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.500495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35751e29-58f8-47fa-8816-03ca429fb764 None admin] RBAC: Authorizing `identity:get_user(user_id=1decd3055af14f42987a890baebb81a7)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:18.501969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35751e29-58f8-47fa-8816-03ca429fb764 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:18.503926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 87/178] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:18 2022] GET /identity/v3/users/1decd3055af14f42987a890baebb81a7 => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:18.505146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4c79c-d1d3-4c0c-9f37-d3b76661d022 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.505425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4c79c-d1d3-4c0c-9f37-d3b76661d022 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.505695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4c79c-d1d3-4c0c-9f37-d3b76661d022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.521667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cee22d-5235-422b-a067-089ece16ecc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:18.523193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cee22d-5235-422b-a067-089ece16ecc4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.523677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cee22d-5235-422b-a067-089ece16ecc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.524348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cee22d-5235-422b-a067-089ece16ecc4 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.525893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cee22d-5235-422b-a067-089ece16ecc4 None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:18.526231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-66d4c79c-d1d3-4c0c-9f37-d3b76661d022 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:18.527434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-66d4c79c-d1d3-4c0c-9f37-d3b76661d022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:18.528074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cee22d-5235-422b-a067-089ece16ecc4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:18.529972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 92/179] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:18 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:18.530958 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cee22d-5235-422b-a067-089ece16ecc4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:18.598512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 88/180] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:18.599777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc22314-dd84-4148-abc7-ed864459ba65 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.600061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc22314-dd84-4148-abc7-ed864459ba65 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.600368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc22314-dd84-4148-abc7-ed864459ba65 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.605181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 89/181] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:18.605181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc22314-dd84-4148-abc7-ed864459ba65 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:18.614676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:18.637513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:18.670122 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cee22d-5235-422b-a067-089ece16ecc4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:18.715534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc22314-dd84-4148-abc7-ed864459ba65 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:18.737528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad90c6-5fde-41f6-a194-06174021cdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:18.738631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad90c6-5fde-41f6-a194-06174021cdcd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.739107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad90c6-5fde-41f6-a194-06174021cdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.739771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad90c6-5fde-41f6-a194-06174021cdcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.759907 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402ab50-f079-4cdf-99e9-8c553e5494b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:18.760742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402ab50-f079-4cdf-99e9-8c553e5494b3 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.761139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402ab50-f079-4cdf-99e9-8c553e5494b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.762126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402ab50-f079-4cdf-99e9-8c553e5494b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.762910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402ab50-f079-4cdf-99e9-8c553e5494b3 None admin] RBAC: Adding query filter params (role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5, scope.project.id=9164929bc082468aa5cb93ca3d057091, user.id=17297aae8d6f4c7b801260350bd22b41) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:18.763344 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402ab50-f079-4cdf-99e9-8c553e5494b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:18.765033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402ab50-f079-4cdf-99e9-8c553e5494b3 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:18.771675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 90/182] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:18 2022] GET /identity/v3/role_assignments?user.id=17297aae8d6f4c7b801260350bd22b41&scope.project.id=9164929bc082468aa5cb93ca3d057091&role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:18.773658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd710d4-1196-4a19-aa69-50ceec05ef9c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.774085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd710d4-1196-4a19-aa69-50ceec05ef9c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.774256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd710d4-1196-4a19-aa69-50ceec05ef9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.795067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ecd710d4-1196-4a19-aa69-50ceec05ef9c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:18.796455 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ecd710d4-1196-4a19-aa69-50ceec05ef9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:18.839312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad90c6-5fde-41f6-a194-06174021cdcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:18.841999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad90c6-5fde-41f6-a194-06174021cdcd None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:18.882633 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 91/183] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:18.883296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd710d4-1196-4a19-aa69-50ceec05ef9c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:18.886075 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:18.961214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:18.988798 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 93/184] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:18.990990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948562c-a6a6-4060-966b-1969e337071e None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:18.992075 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948562c-a6a6-4060-966b-1969e337071e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:18.992857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948562c-a6a6-4060-966b-1969e337071e None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:18.995096 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 94/185] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:18.996694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948562c-a6a6-4060-966b-1969e337071e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:18.998558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd710d4-1196-4a19-aa69-50ceec05ef9c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:19.038752 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ff5b3-90b9-49ea-89a1-967db276d877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:19.039528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ff5b3-90b9-49ea-89a1-967db276d877 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:19.039806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ff5b3-90b9-49ea-89a1-967db276d877 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:19.040077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ff5b3-90b9-49ea-89a1-967db276d877 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:19.051956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ff5b3-90b9-49ea-89a1-967db276d877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-7a0ff5b3-90b9-49ea-89a1-967db276d877 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:19.058257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:19.059138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:19.059933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 92/186] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:18:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:19.060566 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c911f-59af-4834-8945-e15f0fc21b28 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:19.061239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c911f-59af-4834-8945-e15f0fc21b28 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:19.061239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c911f-59af-4834-8945-e15f0fc21b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:19.082158 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b95c911f-59af-4834-8945-e15f0fc21b28 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:19.083343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b95c911f-59af-4834-8945-e15f0fc21b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:19.125186 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948562c-a6a6-4060-966b-1969e337071e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:19.155452 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:19.167922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935232ee-04b2-4133-8dd4-621e7d6626b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:19.168574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935232ee-04b2-4133-8dd4-621e7d6626b9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:19.168909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935232ee-04b2-4133-8dd4-621e7d6626b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:19.169190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935232ee-04b2-4133-8dd4-621e7d6626b9 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:19.170450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935232ee-04b2-4133-8dd4-621e7d6626b9 None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:19.170562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 93/187] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:19.171359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c911f-59af-4834-8945-e15f0fc21b28 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:19.172340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935232ee-04b2-4133-8dd4-621e7d6626b9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:19.175463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 95/188] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:18 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:19.177817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935232ee-04b2-4133-8dd4-621e7d6626b9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:19.241146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:19.282660 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c911f-59af-4834-8945-e15f0fc21b28 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:19.297821 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935232ee-04b2-4133-8dd4-621e7d6626b9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:19.319798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f966f5c-d2db-4f17-aec9-6e2b6fbbbb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:19.320643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f966f5c-d2db-4f17-aec9-6e2b6fbbbb8b None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:19.320954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f966f5c-d2db-4f17-aec9-6e2b6fbbbb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:19.321343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f966f5c-d2db-4f17-aec9-6e2b6fbbbb8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:19.331647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751ee23-bb2f-4039-912f-f1090e570b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:19.332285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751ee23-bb2f-4039-912f-f1090e570b32 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:19.332574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751ee23-bb2f-4039-912f-f1090e570b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:19.332669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f966f5c-d2db-4f17-aec9-6e2b6fbbbb8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:19.332879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751ee23-bb2f-4039-912f-f1090e570b32 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:19.333533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751ee23-bb2f-4039-912f-f1090e570b32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:19.333836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751ee23-bb2f-4039-912f-f1090e570b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:19.335887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751ee23-bb2f-4039-912f-f1090e570b32 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-6f966f5c-d2db-4f17-aec9-6e2b6fbbbb8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:19.338361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:19.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:19.339956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 94/189] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:19.340568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f966f5c-d2db-4f17-aec9-6e2b6fbbbb8b None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:19.354089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 96/190] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:19.354089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751ee23-bb2f-4039-912f-f1090e570b32 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:19.401541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:19.449950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f966f5c-d2db-4f17-aec9-6e2b6fbbbb8b None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:19.453188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751ee23-bb2f-4039-912f-f1090e570b32 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:19.464028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:19.486238 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f739ae3-2a07-44f8-92b7-5f28a3b1c7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:19.486844 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f739ae3-2a07-44f8-92b7-5f28a3b1c7ce None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:19.487123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f739ae3-2a07-44f8-92b7-5f28a3b1c7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:19.487394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f739ae3-2a07-44f8-92b7-5f28a3b1c7ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:19.488115 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28595a99-28e0-41e8-af6a-1e508351d1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:19.488221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f739ae3-2a07-44f8-92b7-5f28a3b1c7ce None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=9164929bc082468aa5cb93ca3d057091, user.id=1decd3055af14f42987a890baebb81a7) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:19.488421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f739ae3-2a07-44f8-92b7-5f28a3b1c7ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:19.489041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28595a99-28e0-41e8-af6a-1e508351d1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:19.489328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28595a99-28e0-41e8-af6a-1e508351d1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:19.489601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28595a99-28e0-41e8-af6a-1e508351d1d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:19.489916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f739ae3-2a07-44f8-92b7-5f28a3b1c7ce None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:19.490708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28595a99-28e0-41e8-af6a-1e508351d1d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:19.492295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28595a99-28e0-41e8-af6a-1e508351d1d4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:19.493845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 97/191] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:19.494607 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28595a99-28e0-41e8-af6a-1e508351d1d4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:19.512796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 95/192] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:19 2022] GET /identity/v3/role_assignments?user.id=1decd3055af14f42987a890baebb81a7&scope.project.id=9164929bc082468aa5cb93ca3d057091&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:19.512796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f739ae3-2a07-44f8-92b7-5f28a3b1c7ce None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:19.615398 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f739ae3-2a07-44f8-92b7-5f28a3b1c7ce None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:19.618116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28595a99-28e0-41e8-af6a-1e508351d1d4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:19.634118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:19.659586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da24df-d3eb-44ac-86b1-1d8b28821f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:19.659762 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989744b-e047-4301-8878-7ee42ffbe2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:19.660218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da24df-d3eb-44ac-86b1-1d8b28821f45 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:19.660324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989744b-e047-4301-8878-7ee42ffbe2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:19.660495 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da24df-d3eb-44ac-86b1-1d8b28821f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:19.660593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989744b-e047-4301-8878-7ee42ffbe2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:19.660797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da24df-d3eb-44ac-86b1-1d8b28821f45 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:19.660901 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989744b-e047-4301-8878-7ee42ffbe2b6 None admin] PATH_INFO: `/v3/users/bb499eddbe004870aa0946730a98721e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:19.661464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da24df-d3eb-44ac-86b1-1d8b28821f45 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:19.661767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da24df-d3eb-44ac-86b1-1d8b28821f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:19.663293 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da24df-d3eb-44ac-86b1-1d8b28821f45 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:19.673404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 98/193] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:19.674579 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da24df-d3eb-44ac-86b1-1d8b28821f45 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:19.677971 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989744b-e047-4301-8878-7ee42ffbe2b6 None admin] RBAC: Authorizing `identity:get_user(user_id=bb499eddbe004870aa0946730a98721e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:19.679810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989744b-e047-4301-8878-7ee42ffbe2b6 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:19.686043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 96/194] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:19 2022] GET /identity/v3/users/bb499eddbe004870aa0946730a98721e => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:19.686043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989744b-e047-4301-8878-7ee42ffbe2b6 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:19.725184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:19.784010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da24df-d3eb-44ac-86b1-1d8b28821f45 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:19.793603 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989744b-e047-4301-8878-7ee42ffbe2b6 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:19.817263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395c4e8-fdd0-4e69-9d39-986e17300b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:19.817902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395c4e8-fdd0-4e69-9d39-986e17300b1f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:19.818185 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395c4e8-fdd0-4e69-9d39-986e17300b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:19.818464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395c4e8-fdd0-4e69-9d39-986e17300b1f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:19.828111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203be4ac-b735-49af-a47d-be8b35f488e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:19.828765 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203be4ac-b735-49af-a47d-be8b35f488e9 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:19.829065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395c4e8-fdd0-4e69-9d39-986e17300b1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:19.829161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203be4ac-b735-49af-a47d-be8b35f488e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:19.829336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203be4ac-b735-49af-a47d-be8b35f488e9 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:19.829981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203be4ac-b735-49af-a47d-be8b35f488e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:19.831428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203be4ac-b735-49af-a47d-be8b35f488e9 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a395c4e8-fdd0-4e69-9d39-986e17300b1f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:19.834276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:19.835685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:19.836444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 99/195] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:19.837486 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395c4e8-fdd0-4e69-9d39-986e17300b1f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:19.853060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 97/196] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:18:19 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:18:19.854202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203be4ac-b735-49af-a47d-be8b35f488e9 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:19.890124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:19.969200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203be4ac-b735-49af-a47d-be8b35f488e9 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:19.969472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:19.973899 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395c4e8-fdd0-4e69-9d39-986e17300b1f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:20.010531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c84be-e094-46cb-aa70-66512c83d08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:20.011140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c84be-e094-46cb-aa70-66512c83d08a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:20.011415 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c84be-e094-46cb-aa70-66512c83d08a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:20.011681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c84be-e094-46cb-aa70-66512c83d08a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:20.019823 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbfd1e-b6d2-4bc3-a931-90121700293e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:20.020459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbfd1e-b6d2-4bc3-a931-90121700293e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:20.020738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbfd1e-b6d2-4bc3-a931-90121700293e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:20.021051 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbfd1e-b6d2-4bc3-a931-90121700293e None admin] PATH_INFO: `/v3/users/bb499eddbe004870aa0946730a98721e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:20.022221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbfd1e-b6d2-4bc3-a931-90121700293e None admin] RBAC: Authorizing `identity:get_user(user_id=bb499eddbe004870aa0946730a98721e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:20.023687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbfd1e-b6d2-4bc3-a931-90121700293e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:20.025605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 100/197] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:19 2022] GET /identity/v3/users/bb499eddbe004870aa0946730a98721e => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:20.026383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbfd1e-b6d2-4bc3-a931-90121700293e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:20.096794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c84be-e094-46cb-aa70-66512c83d08a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:20.096794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c84be-e094-46cb-aa70-66512c83d08a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:20.138787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:20.144345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbfd1e-b6d2-4bc3-a931-90121700293e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:20.187224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f6ca7-3d55-44cc-abfe-ceee9236b452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:20.187847 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f6ca7-3d55-44cc-abfe-ceee9236b452 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:20.188127 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f6ca7-3d55-44cc-abfe-ceee9236b452 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:20.188407 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f6ca7-3d55-44cc-abfe-ceee9236b452 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:20.189110 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f6ca7-3d55-44cc-abfe-ceee9236b452 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:20.189408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f6ca7-3d55-44cc-abfe-ceee9236b452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:20.190792 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f6ca7-3d55-44cc-abfe-ceee9236b452 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:20.203601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 101/198] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:20.209692 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f6ca7-3d55-44cc-abfe-ceee9236b452 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:20.220821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 98/199] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:20.221683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c84be-e094-46cb-aa70-66512c83d08a None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:20.255220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:20.329544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c84be-e094-46cb-aa70-66512c83d08a None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:20.361538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f6ca7-3d55-44cc-abfe-ceee9236b452 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:20.365184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:20.373176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6de6ba-ebdc-4b00-9a43-3ed2175e3cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:20.374607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6de6ba-ebdc-4b00-9a43-3ed2175e3cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:20.375098 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6de6ba-ebdc-4b00-9a43-3ed2175e3cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:20.376016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6de6ba-ebdc-4b00-9a43-3ed2175e3cf6 None admin] PATH_INFO: `/v3/users/ca9e4c836f1b4113ba4c42c043f594d0` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:20.377709 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6de6ba-ebdc-4b00-9a43-3ed2175e3cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=ca9e4c836f1b4113ba4c42c043f594d0)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:20.380195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6de6ba-ebdc-4b00-9a43-3ed2175e3cf6 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:20.388445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 99/200] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:20 2022] GET /identity/v3/users/ca9e4c836f1b4113ba4c42c043f594d0 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:20.388445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6de6ba-ebdc-4b00-9a43-3ed2175e3cf6 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:20.406978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1983ea5-aeae-4785-88b1-151f4a0f8f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:20.407530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1983ea5-aeae-4785-88b1-151f4a0f8f8b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:20.407782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1983ea5-aeae-4785-88b1-151f4a0f8f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:20.408061 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1983ea5-aeae-4785-88b1-151f4a0f8f8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:20.484369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1983ea5-aeae-4785-88b1-151f4a0f8f8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:20.486001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1983ea5-aeae-4785-88b1-151f4a0f8f8b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:20.526124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6de6ba-ebdc-4b00-9a43-3ed2175e3cf6 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:20.532348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:20.570754 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be587551-e678-4a5f-8777-6a330cffce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:20.572339 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be587551-e678-4a5f-8777-6a330cffce51 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:20.572866 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be587551-e678-4a5f-8777-6a330cffce51 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:20.573343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be587551-e678-4a5f-8777-6a330cffce51 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:20.575168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be587551-e678-4a5f-8777-6a330cffce51 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:20.577417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be587551-e678-4a5f-8777-6a330cffce51 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:20.579885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 100/201] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:20 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:20.581418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be587551-e678-4a5f-8777-6a330cffce51 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:20.598156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 102/202] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:20.600025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1983ea5-aeae-4785-88b1-151f4a0f8f8b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:20.667074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:20.713288 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be587551-e678-4a5f-8777-6a330cffce51 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:20.736275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1983ea5-aeae-4785-88b1-151f4a0f8f8b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:20.749735 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd2d48-9530-4025-aac2-f669074281c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:20.750324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd2d48-9530-4025-aac2-f669074281c4 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:20.750603 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd2d48-9530-4025-aac2-f669074281c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:20.750925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd2d48-9530-4025-aac2-f669074281c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:20.777353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:20.786546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99aee3-a2a1-4a17-b29a-0b2ca46c5667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:20.787457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99aee3-a2a1-4a17-b29a-0b2ca46c5667 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:20.788210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99aee3-a2a1-4a17-b29a-0b2ca46c5667 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:20.789574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99aee3-a2a1-4a17-b29a-0b2ca46c5667 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:20.790497 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99aee3-a2a1-4a17-b29a-0b2ca46c5667 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=cbda193ce0194e0294b307a9296fa907, user.id=bb499eddbe004870aa0946730a98721e) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:20.790991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99aee3-a2a1-4a17-b29a-0b2ca46c5667 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:20.792627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99aee3-a2a1-4a17-b29a-0b2ca46c5667 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:20.800639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 103/203] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:20 2022] GET /identity/v3/role_assignments?user.id=bb499eddbe004870aa0946730a98721e&scope.project.id=cbda193ce0194e0294b307a9296fa907&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:20.802484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99aee3-a2a1-4a17-b29a-0b2ca46c5667 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:20.829429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd2d48-9530-4025-aac2-f669074281c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:20.831129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd2d48-9530-4025-aac2-f669074281c4 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:20.925704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:20.941866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99aee3-a2a1-4a17-b29a-0b2ca46c5667 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:20.990553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587571df-bec8-4dac-bea0-0ff778c745cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:20.991933 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587571df-bec8-4dac-bea0-0ff778c745cc None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:20.992215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587571df-bec8-4dac-bea0-0ff778c745cc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:20.992487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587571df-bec8-4dac-bea0-0ff778c745cc None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:20.997163 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587571df-bec8-4dac-bea0-0ff778c745cc None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:20.998662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587571df-bec8-4dac-bea0-0ff778c745cc None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:21.004923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 104/204] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:20 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:21.007585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587571df-bec8-4dac-bea0-0ff778c745cc None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:21.027176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 101/205] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:21.033432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd2d48-9530-4025-aac2-f669074281c4 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:21.082384 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:21.128262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587571df-bec8-4dac-bea0-0ff778c745cc None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:21.140718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd2d48-9530-4025-aac2-f669074281c4 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:21.166943 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3da2c3-0ac1-485f-80bf-49f476ef0291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:21.168105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3da2c3-0ac1-485f-80bf-49f476ef0291 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:21.168614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3da2c3-0ac1-485f-80bf-49f476ef0291 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:21.169190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3da2c3-0ac1-485f-80bf-49f476ef0291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:21.170153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3da2c3-0ac1-485f-80bf-49f476ef0291 None admin] RBAC: Adding query filter params (role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5, scope.project.id=cbda193ce0194e0294b307a9296fa907, user.id=bb499eddbe004870aa0946730a98721e) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:21.170748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3da2c3-0ac1-485f-80bf-49f476ef0291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:21.172691 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3da2c3-0ac1-485f-80bf-49f476ef0291 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:21.177192 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96724da7-07a1-469a-b3b7-c4f80c78b795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:21.177762 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96724da7-07a1-469a-b3b7-c4f80c78b795 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:21.178046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96724da7-07a1-469a-b3b7-c4f80c78b795 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:21.178322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96724da7-07a1-469a-b3b7-c4f80c78b795 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:21.180540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96724da7-07a1-469a-b3b7-c4f80c78b795 None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:21.181666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 105/206] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:21 2022] GET /identity/v3/role_assignments?user.id=bb499eddbe004870aa0946730a98721e&scope.project.id=cbda193ce0194e0294b307a9296fa907&role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:21.182361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96724da7-07a1-469a-b3b7-c4f80c78b795 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:21.189173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 102/207] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:21 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:21.191486 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3da2c3-0ac1-485f-80bf-49f476ef0291 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:21.317775 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3da2c3-0ac1-485f-80bf-49f476ef0291 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:21.353438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:21.363251 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f7def-d4e9-45c3-87ec-129f26040c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:21.364127 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f7def-d4e9-45c3-87ec-129f26040c7e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:21.364648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f7def-d4e9-45c3-87ec-129f26040c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:21.365132 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f7def-d4e9-45c3-87ec-129f26040c7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:21.366617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f7def-d4e9-45c3-87ec-129f26040c7e None admin] RBAC: Adding query filter params (role.id=732934b017114352b11d54cb795862b1, scope.project.id=9164929bc082468aa5cb93ca3d057091, user.id=ca9e4c836f1b4113ba4c42c043f594d0) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:21.367135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f7def-d4e9-45c3-87ec-129f26040c7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:21.368863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f7def-d4e9-45c3-87ec-129f26040c7e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:21.375175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 106/208] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:21 2022] GET /identity/v3/role_assignments?user.id=ca9e4c836f1b4113ba4c42c043f594d0&scope.project.id=9164929bc082468aa5cb93ca3d057091&role.id=732934b017114352b11d54cb795862b1 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:22.120164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0759a2-1ee8-43cd-92c5-9f234f53f337 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:22.121200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0759a2-1ee8-43cd-92c5-9f234f53f337 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:22.121682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0759a2-1ee8-43cd-92c5-9f234f53f337 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:22.123349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 103/209] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:22.129446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5934df-3bb1-4200-9f51-b790963a1e12 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:22.130279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5934df-3bb1-4200-9f51-b790963a1e12 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:22.130610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6261d2-a789-4c0d-bb3a-cbf0c9e2ffe1 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:22.130904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6261d2-a789-4c0d-bb3a-cbf0c9e2ffe1 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:22.131164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5934df-3bb1-4200-9f51-b790963a1e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:22.131261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6261d2-a789-4c0d-bb3a-cbf0c9e2ffe1 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:22.133049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 104/210] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:22.139152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267d914-35d1-4333-94cb-dcebad563196 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:22.139670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267d914-35d1-4333-94cb-dcebad563196 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:22.140332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267d914-35d1-4333-94cb-dcebad563196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:22.153690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7a5934df-3bb1-4200-9f51-b790963a1e12 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:22.155040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7a5934df-3bb1-4200-9f51-b790963a1e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:22.162546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-2267d914-35d1-4333-94cb-dcebad563196 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:22.164703 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2267d914-35d1-4333-94cb-dcebad563196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:22.199567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:22.237137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 107/211] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:22.247099 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 105/212] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:22.511650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5934df-3bb1-4200-9f51-b790963a1e12 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:22.517082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267d914-35d1-4333-94cb-dcebad563196 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:22.621505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5934df-3bb1-4200-9f51-b790963a1e12 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:22.648208 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267d914-35d1-4333-94cb-dcebad563196 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:22.650286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:22.674087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65514abc-3a9b-4a26-b243-13439c7275de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:22.675064 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65514abc-3a9b-4a26-b243-13439c7275de None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:22.675536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65514abc-3a9b-4a26-b243-13439c7275de None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:22.675994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65514abc-3a9b-4a26-b243-13439c7275de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:22.677437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65514abc-3a9b-4a26-b243-13439c7275de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:22.679642 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65514abc-3a9b-4a26-b243-13439c7275de None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:22.686480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 108/213] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:22.688148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65514abc-3a9b-4a26-b243-13439c7275de None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:22.692789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d1749-f3d6-442c-b264-4ef286ad27cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:22.693434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d1749-f3d6-442c-b264-4ef286ad27cc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:22.693713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d1749-f3d6-442c-b264-4ef286ad27cc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:22.693983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d1749-f3d6-442c-b264-4ef286ad27cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:22.704525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d1749-f3d6-442c-b264-4ef286ad27cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-a74d1749-f3d6-442c-b264-4ef286ad27cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:22.709311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:22.710263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:22.711461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:22.711461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:22.711461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:22.711461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:22.711461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:22.711461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:22.711461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:22.711461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:22.711461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:22.711461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:22.711461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:22.711461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:22.715510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 106/214] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:18:22.716534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d1749-f3d6-442c-b264-4ef286ad27cc None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:22.750088 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:22.818764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65514abc-3a9b-4a26-b243-13439c7275de None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:22.858360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d1749-f3d6-442c-b264-4ef286ad27cc None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:22.869513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21273c3c-1799-4ee6-8454-d3fb12e62184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:22.870874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21273c3c-1799-4ee6-8454-d3fb12e62184 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:22.871354 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21273c3c-1799-4ee6-8454-d3fb12e62184 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:22.872210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21273c3c-1799-4ee6-8454-d3fb12e62184 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:22.873366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21273c3c-1799-4ee6-8454-d3fb12e62184 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:22.875773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21273c3c-1799-4ee6-8454-d3fb12e62184 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:22.898413 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee5c1a-fe89-4d84-956c-72fa5509c22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:22.899294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee5c1a-fe89-4d84-956c-72fa5509c22e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:22.899581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee5c1a-fe89-4d84-956c-72fa5509c22e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:22.899894 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee5c1a-fe89-4d84-956c-72fa5509c22e None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:22.900627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee5c1a-fe89-4d84-956c-72fa5509c22e None admin] RBAC: Adding query filter params (name=member) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:22.901053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee5c1a-fe89-4d84-956c-72fa5509c22e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:22.902936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee5c1a-fe89-4d84-956c-72fa5509c22e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:22.906224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 109/215] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:18:22 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:18:22.917487 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 107/216] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:22.923287 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21273c3c-1799-4ee6-8454-d3fb12e62184 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:22.978895 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:22.991946 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cfdf7-8f8c-4e60-b45c-4cf4cbacfded None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:22.992277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cfdf7-8f8c-4e60-b45c-4cf4cbacfded None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:22.992702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cfdf7-8f8c-4e60-b45c-4cf4cbacfded None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:22.994580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 108/217] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:23.000838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ecddd-d56b-469d-afe8-49221a664c78 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:23.001127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ecddd-d56b-469d-afe8-49221a664c78 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:23.001406 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ecddd-d56b-469d-afe8-49221a664c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:23.020146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-8e8ecddd-d56b-469d-afe8-49221a664c78 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:23.021559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8e8ecddd-d56b-469d-afe8-49221a664c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:23.047202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21273c3c-1799-4ee6-8454-d3fb12e62184 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:23.088335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aba8b-6a17-4005-b237-dc4277919b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:23.088941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aba8b-6a17-4005-b237-dc4277919b7d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:23.089255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aba8b-6a17-4005-b237-dc4277919b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:23.089615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aba8b-6a17-4005-b237-dc4277919b7d None admin] PATH_INFO: `/v3/users/36bee95cf1dc4a98bce595fc5b0dee95` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:23.108290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aba8b-6a17-4005-b237-dc4277919b7d None admin] RBAC: Authorizing `identity:get_user(user_id=36bee95cf1dc4a98bce595fc5b0dee95)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:23.109938 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aba8b-6a17-4005-b237-dc4277919b7d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:23.111716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 110/218] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:22 2022] GET /identity/v3/users/36bee95cf1dc4a98bce595fc5b0dee95 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:23.115360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 109/219] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:23.121979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aba8b-6a17-4005-b237-dc4277919b7d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:23.232342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aba8b-6a17-4005-b237-dc4277919b7d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:23.234105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:23.270796 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bb11c-53e8-4097-8490-422f1266749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:23.271505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bb11c-53e8-4097-8490-422f1266749b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:23.271852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bb11c-53e8-4097-8490-422f1266749b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:23.272845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bb11c-53e8-4097-8490-422f1266749b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:23.300325 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ec8a1-c840-489d-92be-b2350d8f69fa None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:23.301258 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ec8a1-c840-489d-92be-b2350d8f69fa None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:23.301748 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ec8a1-c840-489d-92be-b2350d8f69fa None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:23.303997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 110/220] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:23.309213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1bdc8-0f4a-4616-9d59-f81dd9fba9a4 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:23.309497 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1bdc8-0f4a-4616-9d59-f81dd9fba9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:23.309771 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1bdc8-0f4a-4616-9d59-f81dd9fba9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:23.334454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-15e1bdc8-0f4a-4616-9d59-f81dd9fba9a4 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:23.337054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-15e1bdc8-0f4a-4616-9d59-f81dd9fba9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:23.351198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:23.358547 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bb11c-53e8-4097-8490-422f1266749b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:23.360298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bb11c-53e8-4097-8490-422f1266749b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:23.417735 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 111/221] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:23.418812 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1bdc8-0f4a-4616-9d59-f81dd9fba9a4 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:23.481922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 111/222] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 16:18:23.493765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bb11c-53e8-4097-8490-422f1266749b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:23.496089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:23.539574 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1bdc8-0f4a-4616-9d59-f81dd9fba9a4 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:23.575191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac865392-38bd-42d7-9140-100486607240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:23.575837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac865392-38bd-42d7-9140-100486607240 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:23.576117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac865392-38bd-42d7-9140-100486607240 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:23.576390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac865392-38bd-42d7-9140-100486607240 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:23.587358 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac865392-38bd-42d7-9140-100486607240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-ac865392-38bd-42d7-9140-100486607240 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:23.592348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:23.593244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:23.593998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 112/223] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:18:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:23.599489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac865392-38bd-42d7-9140-100486607240 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:23.619825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:23.712283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac865392-38bd-42d7-9140-100486607240 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:23.715257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bb11c-53e8-4097-8490-422f1266749b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:23.752336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9b4ba-01ed-4135-b54b-3eac1f62b787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:23.753123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9b4ba-01ed-4135-b54b-3eac1f62b787 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:23.753410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9b4ba-01ed-4135-b54b-3eac1f62b787 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:23.753681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9b4ba-01ed-4135-b54b-3eac1f62b787 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:23.754166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac6916-45cb-41a0-b71d-1d17990e42a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:23.754388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9b4ba-01ed-4135-b54b-3eac1f62b787 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:23.754648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9b4ba-01ed-4135-b54b-3eac1f62b787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:23.755412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac6916-45cb-41a0-b71d-1d17990e42a4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:23.755885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac6916-45cb-41a0-b71d-1d17990e42a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:23.756068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9b4ba-01ed-4135-b54b-3eac1f62b787 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:23.756591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac6916-45cb-41a0-b71d-1d17990e42a4 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:23.758503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac6916-45cb-41a0-b71d-1d17990e42a4 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:23.760052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac6916-45cb-41a0-b71d-1d17990e42a4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:23.762143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 112/224] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:23 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:23.767401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac6916-45cb-41a0-b71d-1d17990e42a4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:23.771329 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 113/225] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:23.772984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9b4ba-01ed-4135-b54b-3eac1f62b787 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:23.843790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:23.884449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac6916-45cb-41a0-b71d-1d17990e42a4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:23.885120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:23.904085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9b4ba-01ed-4135-b54b-3eac1f62b787 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:23.931149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92970a-50d1-4ec1-a3c2-8782060d473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:23.932331 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92970a-50d1-4ec1-a3c2-8782060d473c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:23.932621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92970a-50d1-4ec1-a3c2-8782060d473c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:23.933198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92970a-50d1-4ec1-a3c2-8782060d473c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:23.947229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92970a-50d1-4ec1-a3c2-8782060d473c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:23.948876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f26c2-1b8b-4789-85c4-c8b9694a276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:23.949651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f26c2-1b8b-4789-85c4-c8b9694a276e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:23.950034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f26c2-1b8b-4789-85c4-c8b9694a276e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:23.950462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f26c2-1b8b-4789-85c4-c8b9694a276e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:23.951304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f26c2-1b8b-4789-85c4-c8b9694a276e None admin] RBAC: Adding query filter params (role.id=ac5a835695db44918d768db1bbb97589, scope.project.id=cbda193ce0194e0294b307a9296fa907, user.id=36bee95cf1dc4a98bce595fc5b0dee95) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:23.951717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f26c2-1b8b-4789-85c4-c8b9694a276e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-6c92970a-50d1-4ec1-a3c2-8782060d473c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:23.951997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:23.952875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:23.953621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 113/226] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:23.954249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92970a-50d1-4ec1-a3c2-8782060d473c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:23.957252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f26c2-1b8b-4789-85c4-c8b9694a276e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:23.964434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 114/227] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:23 2022] GET /identity/v3/role_assignments?user.id=36bee95cf1dc4a98bce595fc5b0dee95&scope.project.id=cbda193ce0194e0294b307a9296fa907&role.id=ac5a835695db44918d768db1bbb97589 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:23.965370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f26c2-1b8b-4789-85c4-c8b9694a276e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:24.075472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f26c2-1b8b-4789-85c4-c8b9694a276e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:24.082953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92970a-50d1-4ec1-a3c2-8782060d473c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:24.117394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:24.124026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9ef3e-f490-4f69-af11-e18c7f5154de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:24.125022 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9ef3e-f490-4f69-af11-e18c7f5154de None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:24.125583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9ef3e-f490-4f69-af11-e18c7f5154de None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:24.126042 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9ef3e-f490-4f69-af11-e18c7f5154de None admin] PATH_INFO: `/v3/users/bb499eddbe004870aa0946730a98721e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:24.127447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9ef3e-f490-4f69-af11-e18c7f5154de None admin] RBAC: Authorizing `identity:get_user(user_id=bb499eddbe004870aa0946730a98721e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:24.127926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:24.129738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9ef3e-f490-4f69-af11-e18c7f5154de None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:24.132448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 114/228] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:23 2022] GET /identity/v3/users/bb499eddbe004870aa0946730a98721e => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:24.134531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9bc11-407b-4895-b4a1-4dfa26a6bc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:24.135574 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9bc11-407b-4895-b4a1-4dfa26a6bc6e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:24.136072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9bc11-407b-4895-b4a1-4dfa26a6bc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:24.136537 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9bc11-407b-4895-b4a1-4dfa26a6bc6e None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:24.137307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9bc11-407b-4895-b4a1-4dfa26a6bc6e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:24.137679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9bc11-407b-4895-b4a1-4dfa26a6bc6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:24.139138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9bc11-407b-4895-b4a1-4dfa26a6bc6e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:24.139371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9ef3e-f490-4f69-af11-e18c7f5154de None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:24.149400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 115/229] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:24.155146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9bc11-407b-4895-b4a1-4dfa26a6bc6e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:24.257398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9ef3e-f490-4f69-af11-e18c7f5154de None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:24.262915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9bc11-407b-4895-b4a1-4dfa26a6bc6e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:24.296854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f442241-1209-420d-aea6-279ba9535aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:24.297599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68195ea4-7525-43db-a5bf-e211eb328023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:24.297788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f442241-1209-420d-aea6-279ba9535aca None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:24.297975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f442241-1209-420d-aea6-279ba9535aca None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:24.298119 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f442241-1209-420d-aea6-279ba9535aca None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:24.298259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68195ea4-7525-43db-a5bf-e211eb328023 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:24.298494 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68195ea4-7525-43db-a5bf-e211eb328023 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:24.298768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68195ea4-7525-43db-a5bf-e211eb328023 None admin] PATH_INFO: `/v3/users/bb499eddbe004870aa0946730a98721e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:24.299373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f442241-1209-420d-aea6-279ba9535aca None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:24.299927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68195ea4-7525-43db-a5bf-e211eb328023 None admin] RBAC: Authorizing `identity:get_user(user_id=bb499eddbe004870aa0946730a98721e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:24.300861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f442241-1209-420d-aea6-279ba9535aca None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:24.301412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68195ea4-7525-43db-a5bf-e211eb328023 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:24.302912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 115/230] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:24 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:24.303257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 116/231] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:24 2022] GET /identity/v3/users/bb499eddbe004870aa0946730a98721e => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:24.308727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f442241-1209-420d-aea6-279ba9535aca None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:24.308965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68195ea4-7525-43db-a5bf-e211eb328023 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:24.375308 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:24.386464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:24.417337 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68195ea4-7525-43db-a5bf-e211eb328023 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:24.426234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f442241-1209-420d-aea6-279ba9535aca None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:24.452299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4742f3-eff3-4d40-940a-3113be2858e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:24.452984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4742f3-eff3-4d40-940a-3113be2858e8 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:24.453277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4742f3-eff3-4d40-940a-3113be2858e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:24.453557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4742f3-eff3-4d40-940a-3113be2858e8 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:24.454819 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4742f3-eff3-4d40-940a-3113be2858e8 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:24.456293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4742f3-eff3-4d40-940a-3113be2858e8 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:24.457971 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 117/232] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:24 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:24.469195 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295ead3-74a2-41c4-be5c-d1b93c611918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:24.469721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4742f3-eff3-4d40-940a-3113be2858e8 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:24.469916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295ead3-74a2-41c4-be5c-d1b93c611918 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:24.470177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295ead3-74a2-41c4-be5c-d1b93c611918 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:24.470410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295ead3-74a2-41c4-be5c-d1b93c611918 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907/users/bb499eddbe004870aa0946730a98721e/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:24.472475 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295ead3-74a2-41c4-be5c-d1b93c611918 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbda193ce0194e0294b307a9296fa907, user_id=bb499eddbe004870aa0946730a98721e, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:24.473974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295ead3-74a2-41c4-be5c-d1b93c611918 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:24.508453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 116/233] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:18:24 2022] PUT /identity/v3/projects/cbda193ce0194e0294b307a9296fa907/users/bb499eddbe004870aa0946730a98721e/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:24.592176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4742f3-eff3-4d40-940a-3113be2858e8 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:24.625903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:24.631981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4011e7d-4928-459b-bda9-2c5817cfbb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:24.632784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4011e7d-4928-459b-bda9-2c5817cfbb56 None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:24.633274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4011e7d-4928-459b-bda9-2c5817cfbb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:24.633799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4011e7d-4928-459b-bda9-2c5817cfbb56 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907/users/bb499eddbe004870aa0946730a98721e/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:24.636417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4011e7d-4928-459b-bda9-2c5817cfbb56 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbda193ce0194e0294b307a9296fa907, user_id=bb499eddbe004870aa0946730a98721e, role_id=f7bd4a6c906a4ad3b6d2f003f9c37bc5)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:24.638020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4011e7d-4928-459b-bda9-2c5817cfbb56 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:24.670849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 118/234] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:18:24 2022] PUT /identity/v3/projects/cbda193ce0194e0294b307a9296fa907/users/bb499eddbe004870aa0946730a98721e/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:25.541743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dacaa-4244-4326-abb3-4c596e097fd9 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:25.542287 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dacaa-4244-4326-abb3-4c596e097fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:25.542287 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dacaa-4244-4326-abb3-4c596e097fd9 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:25.543900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 117/235] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:25.559385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312a6ec-fe52-449f-b2d6-47a03c4f6258 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:25.559887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2540cae-154b-4f1d-b683-d63fbe3dc718 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:25.559966 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312a6ec-fe52-449f-b2d6-47a03c4f6258 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:25.560043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2540cae-154b-4f1d-b683-d63fbe3dc718 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:25.560109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312a6ec-fe52-449f-b2d6-47a03c4f6258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:25.560174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2540cae-154b-4f1d-b683-d63fbe3dc718 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:25.561611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 118/236] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:18:25.566562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a335f-8b0e-4071-9d42-e221848bd330 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:25.566854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a335f-8b0e-4071-9d42-e221848bd330 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:25.567135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a335f-8b0e-4071-9d42-e221848bd330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:25.591536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d312a6ec-fe52-449f-b2d6-47a03c4f6258 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:25.592725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d312a6ec-fe52-449f-b2d6-47a03c4f6258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:25.598377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3f1a335f-8b0e-4071-9d42-e221848bd330 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:25.599522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3f1a335f-8b0e-4071-9d42-e221848bd330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:25.695493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 119/237] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:25.718302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 119/238] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:26.098490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312a6ec-fe52-449f-b2d6-47a03c4f6258 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:26.166652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a335f-8b0e-4071-9d42-e221848bd330 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:26.246693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312a6ec-fe52-449f-b2d6-47a03c4f6258 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:26.257069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:26.281836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:26.309717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b23f48-2a82-4ed2-95fc-41298c3a2f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:26.310680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b23f48-2a82-4ed2-95fc-41298c3a2f9f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:26.311150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b23f48-2a82-4ed2-95fc-41298c3a2f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:26.311605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b23f48-2a82-4ed2-95fc-41298c3a2f9f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:26.331073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b23f48-2a82-4ed2-95fc-41298c3a2f9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-15b23f48-2a82-4ed2-95fc-41298c3a2f9f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:26.336716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:26.337569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:26.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:26.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:26.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:26.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:26.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:26.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:26.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:26.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:26.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:26.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:26.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:26.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:26.342458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 120/239] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:26.345487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a335f-8b0e-4071-9d42-e221848bd330 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:26.348830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b23f48-2a82-4ed2-95fc-41298c3a2f9f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:26.414145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a96c6e-47d6-4a6c-b967-500fdf26151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:26.419742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a96c6e-47d6-4a6c-b967-500fdf26151e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:26.420260 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a96c6e-47d6-4a6c-b967-500fdf26151e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:26.420727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a96c6e-47d6-4a6c-b967-500fdf26151e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:26.422165 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a96c6e-47d6-4a6c-b967-500fdf26151e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:26.423843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a96c6e-47d6-4a6c-b967-500fdf26151e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:26.433324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 120/240] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:26.443003 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a96c6e-47d6-4a6c-b967-500fdf26151e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:26.536365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b23f48-2a82-4ed2-95fc-41298c3a2f9f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:26.572142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21f4fe-8193-40b8-84bf-ca9be02a44ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:26.572815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f4fe-8193-40b8-84bf-ca9be02a44ec None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:26.572815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f4fe-8193-40b8-84bf-ca9be02a44ec None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:26.573078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f4fe-8193-40b8-84bf-ca9be02a44ec None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:26.573903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21f4fe-8193-40b8-84bf-ca9be02a44ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:26.574343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21f4fe-8193-40b8-84bf-ca9be02a44ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:26.575877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21f4fe-8193-40b8-84bf-ca9be02a44ec None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:26.585444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 121/241] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:26.586892 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc5a83-605c-4fcd-86fc-b27af2f1d651 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:26.587299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc5a83-605c-4fcd-86fc-b27af2f1d651 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:26.587690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc5a83-605c-4fcd-86fc-b27af2f1d651 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:26.589068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 122/242] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:26.590681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc5a83-605c-4fcd-86fc-b27af2f1d651 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:26.609934 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a96c6e-47d6-4a6c-b967-500fdf26151e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:26.637165 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:26.650740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:26.658119 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678dfe9-a731-4b28-b052-8b23ef3d24a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:26.658748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678dfe9-a731-4b28-b052-8b23ef3d24a8 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:26.659028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678dfe9-a731-4b28-b052-8b23ef3d24a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:26.659302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678dfe9-a731-4b28-b052-8b23ef3d24a8 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:26.660212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678dfe9-a731-4b28-b052-8b23ef3d24a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:26.661715 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678dfe9-a731-4b28-b052-8b23ef3d24a8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:26.687945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 121/243] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:18:26 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:18:26.689622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbba33-0545-4f1f-8d4f-ce52284facfc None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:26.690040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbba33-0545-4f1f-8d4f-ce52284facfc None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:26.690464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbba33-0545-4f1f-8d4f-ce52284facfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:26.729859 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c6fbba33-0545-4f1f-8d4f-ce52284facfc None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:26.734137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c6fbba33-0545-4f1f-8d4f-ce52284facfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:26.802049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc5a83-605c-4fcd-86fc-b27af2f1d651 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:26.838065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 122/244] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:26.865598 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900b096-636b-4736-81c7-482c68b16bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:26.867104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b096-636b-4736-81c7-482c68b16bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:26.867527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b096-636b-4736-81c7-482c68b16bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:26.867940 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b096-636b-4736-81c7-482c68b16bb0 None admin] PATH_INFO: `/v3/users/179267ebd4f143df9c0ef11de3c906be` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:26.883279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900b096-636b-4736-81c7-482c68b16bb0 None admin] RBAC: Authorizing `identity:get_user(user_id=179267ebd4f143df9c0ef11de3c906be)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:26.885036 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900b096-636b-4736-81c7-482c68b16bb0 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:26.886974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 123/245] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:26 2022] GET /identity/v3/users/179267ebd4f143df9c0ef11de3c906be => generated 297 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:26.892521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbba33-0545-4f1f-8d4f-ce52284facfc None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:26.945068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf199c-dc08-4dbc-a47c-8ae9af78177c None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:26.945654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf199c-dc08-4dbc-a47c-8ae9af78177c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:26.946071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf199c-dc08-4dbc-a47c-8ae9af78177c None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:26.947916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 124/246] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:18:26.952679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f023069-8751-450b-a6f4-1803f2cef55b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:26.953227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f023069-8751-450b-a6f4-1803f2cef55b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:26.953698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f023069-8751-450b-a6f4-1803f2cef55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:26.981272 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-8f023069-8751-450b-a6f4-1803f2cef55b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:26.981272 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8f023069-8751-450b-a6f4-1803f2cef55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:27.026749 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbba33-0545-4f1f-8d4f-ce52284facfc None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:27.042789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:27.068693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 125/247] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:27.070629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54309263-bea8-424d-a5d3-c635377148e9 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:27.070918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54309263-bea8-424d-a5d3-c635377148e9 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:27.071198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54309263-bea8-424d-a5d3-c635377148e9 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:27.072704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 126/248] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:27.077958 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1947af3-b27d-45b7-ad2a-af1b6ae944ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:27.078115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c8afc-ac76-4761-bce7-b84d6ef71d95 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:27.078342 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c8afc-ac76-4761-bce7-b84d6ef71d95 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:27.078583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1947af3-b27d-45b7-ad2a-af1b6ae944ae None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:27.078676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c8afc-ac76-4761-bce7-b84d6ef71d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:27.078864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1947af3-b27d-45b7-ad2a-af1b6ae944ae None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:27.079139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1947af3-b27d-45b7-ad2a-af1b6ae944ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:27.098493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-8d9c8afc-ac76-4761-bce7-b84d6ef71d95 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:27.099697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8d9c8afc-ac76-4761-bce7-b84d6ef71d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:27.130642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:27.184690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 127/249] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:27.185404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c8afc-ac76-4761-bce7-b84d6ef71d95 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:27.192561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1947af3-b27d-45b7-ad2a-af1b6ae944ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:27.194176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1947af3-b27d-45b7-ad2a-af1b6ae944ae None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:27.290299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c8afc-ac76-4761-bce7-b84d6ef71d95 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:27.299872 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 123/250] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 16:18:27.305215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1947af3-b27d-45b7-ad2a-af1b6ae944ae None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:27.316855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:27.327967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6032b50-d6ce-40db-9c54-5b5baa97027f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:27.328609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6032b50-d6ce-40db-9c54-5b5baa97027f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:27.328922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6032b50-d6ce-40db-9c54-5b5baa97027f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:27.329194 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6032b50-d6ce-40db-9c54-5b5baa97027f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:27.341058 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6032b50-d6ce-40db-9c54-5b5baa97027f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-e6032b50-d6ce-40db-9c54-5b5baa97027f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:27.345883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:27.346948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:27.347908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 128/251] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:27.355450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6032b50-d6ce-40db-9c54-5b5baa97027f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:27.379064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:27.419279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1947af3-b27d-45b7-ad2a-af1b6ae944ae None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:27.463679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6032b50-d6ce-40db-9c54-5b5baa97027f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:27.464683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7cf01-0279-42b8-a20b-03d97fe9ee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:27.465589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7cf01-0279-42b8-a20b-03d97fe9ee3b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:27.466073 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7cf01-0279-42b8-a20b-03d97fe9ee3b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:27.466565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7cf01-0279-42b8-a20b-03d97fe9ee3b None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:27.467952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7cf01-0279-42b8-a20b-03d97fe9ee3b None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:27.469657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7cf01-0279-42b8-a20b-03d97fe9ee3b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:27.471508 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 124/252] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:27 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:27.477760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7cf01-0279-42b8-a20b-03d97fe9ee3b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:27.500855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f86173-deb0-44f7-b729-353beff79021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:27.501629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f86173-deb0-44f7-b729-353beff79021 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:27.502015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f86173-deb0-44f7-b729-353beff79021 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:27.502388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f86173-deb0-44f7-b729-353beff79021 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:27.503228 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f86173-deb0-44f7-b729-353beff79021 None admin] RBAC: Adding query filter params (name=member) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:27.503561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f86173-deb0-44f7-b729-353beff79021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:27.505064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f86173-deb0-44f7-b729-353beff79021 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:27.514948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 129/253] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:27.521554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f86173-deb0-44f7-b729-353beff79021 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:27.600436 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:27.640206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7cf01-0279-42b8-a20b-03d97fe9ee3b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:27.669529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:27.688995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beee8e3-aa7f-4ec5-a551-486943d090e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:27.689778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beee8e3-aa7f-4ec5-a551-486943d090e0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:27.690190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beee8e3-aa7f-4ec5-a551-486943d090e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:27.690595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beee8e3-aa7f-4ec5-a551-486943d090e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:27.691471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beee8e3-aa7f-4ec5-a551-486943d090e0 None admin] RBAC: Adding query filter params (role.id=732934b017114352b11d54cb795862b1, scope.project.id=cbda193ce0194e0294b307a9296fa907, user.id=179267ebd4f143df9c0ef11de3c906be) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:27.691661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f86173-deb0-44f7-b729-353beff79021 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:27.691997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beee8e3-aa7f-4ec5-a551-486943d090e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:27.693869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beee8e3-aa7f-4ec5-a551-486943d090e0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:27.704915 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 125/254] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:27 2022] GET /identity/v3/role_assignments?user.id=179267ebd4f143df9c0ef11de3c906be&scope.project.id=cbda193ce0194e0294b307a9296fa907&role.id=732934b017114352b11d54cb795862b1 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:27.706081 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beee8e3-aa7f-4ec5-a551-486943d090e0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:27.801507 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc398-08dc-4e8b-b6ee-451a22e2610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:27.802165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc398-08dc-4e8b-b6ee-451a22e2610d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:27.802447 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc398-08dc-4e8b-b6ee-451a22e2610d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:27.802721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc398-08dc-4e8b-b6ee-451a22e2610d None admin] PATH_INFO: `/v3/users/36bee95cf1dc4a98bce595fc5b0dee95` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:27.804459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfc398-08dc-4e8b-b6ee-451a22e2610d None admin] RBAC: Authorizing `identity:get_user(user_id=36bee95cf1dc4a98bce595fc5b0dee95)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:27.807957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfc398-08dc-4e8b-b6ee-451a22e2610d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:27.807957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 130/255] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:27 2022] GET /identity/v3/users/36bee95cf1dc4a98bce595fc5b0dee95 => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:27.810299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc398-08dc-4e8b-b6ee-451a22e2610d None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:27.846957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beee8e3-aa7f-4ec5-a551-486943d090e0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:27.922581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017553c-049a-4aee-aec2-f083fc132a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:27.923608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017553c-049a-4aee-aec2-f083fc132a26 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:27.924100 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017553c-049a-4aee-aec2-f083fc132a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:27.924580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017553c-049a-4aee-aec2-f083fc132a26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:27.937418 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017553c-049a-4aee-aec2-f083fc132a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-f017553c-049a-4aee-aec2-f083fc132a26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:27.944040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:27.944916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:27.945678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:27.945678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:27.945678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:27.945678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:27.945678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:27.945678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:27.945678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:27.945678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:27.945678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:27.945678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:27.945678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:27.945678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:27.948148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 126/256] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:18:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:27.949303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017553c-049a-4aee-aec2-f083fc132a26 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:27.952234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:28.021168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc398-08dc-4e8b-b6ee-451a22e2610d None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:28.058673 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74175743-81ec-4c37-be12-876990b8df43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:28.059665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74175743-81ec-4c37-be12-876990b8df43 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:28.060235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74175743-81ec-4c37-be12-876990b8df43 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:28.060768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74175743-81ec-4c37-be12-876990b8df43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:28.071913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74175743-81ec-4c37-be12-876990b8df43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-74175743-81ec-4c37-be12-876990b8df43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:28.077623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:28.078465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:28.079217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:28.079217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:28.079217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:28.079217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:28.079217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:28.079217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:28.079217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:28.079217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:28.079217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:28.079217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:28.079217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:28.079217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:28.081922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 131/257] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:28.083130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74175743-81ec-4c37-be12-876990b8df43 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:28.096571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:28.155297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017553c-049a-4aee-aec2-f083fc132a26 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:28.225000 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a16ad-0669-4bfa-89e1-5fd1158332e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:28.227335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a16ad-0669-4bfa-89e1-5fd1158332e3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:28.228041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a16ad-0669-4bfa-89e1-5fd1158332e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:28.230235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a16ad-0669-4bfa-89e1-5fd1158332e3 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:28.231620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a16ad-0669-4bfa-89e1-5fd1158332e3 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:28.237763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a16ad-0669-4bfa-89e1-5fd1158332e3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:28.239738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 127/258] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:27 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:28.241605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a16ad-0669-4bfa-89e1-5fd1158332e3 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:28.262273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74175743-81ec-4c37-be12-876990b8df43 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:28.303992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d32fe-445c-4b94-9efd-69d9da6fea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:28.304989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d32fe-445c-4b94-9efd-69d9da6fea72 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:28.305503 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d32fe-445c-4b94-9efd-69d9da6fea72 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:28.305987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d32fe-445c-4b94-9efd-69d9da6fea72 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:28.306877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d32fe-445c-4b94-9efd-69d9da6fea72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:28.307440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d32fe-445c-4b94-9efd-69d9da6fea72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:28.309040 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d32fe-445c-4b94-9efd-69d9da6fea72 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:28.318010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 132/259] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:28.319164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d32fe-445c-4b94-9efd-69d9da6fea72 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:28.383969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:28.447019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a16ad-0669-4bfa-89e1-5fd1158332e3 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:28.461402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:28.469851 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d32fe-445c-4b94-9efd-69d9da6fea72 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:28.504777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d844a-5a25-4bb7-bcb8-a5f6e5097f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:28.505478 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d844a-5a25-4bb7-bcb8-a5f6e5097f78 None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:28.505754 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d844a-5a25-4bb7-bcb8-a5f6e5097f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:28.506030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d844a-5a25-4bb7-bcb8-a5f6e5097f78 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907/users/36bee95cf1dc4a98bce595fc5b0dee95/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:28.508211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d844a-5a25-4bb7-bcb8-a5f6e5097f78 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbda193ce0194e0294b307a9296fa907, user_id=36bee95cf1dc4a98bce595fc5b0dee95, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:28.509745 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d844a-5a25-4bb7-bcb8-a5f6e5097f78 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:28.533723 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 133/260] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:18:28 2022] PUT /identity/v3/projects/cbda193ce0194e0294b307a9296fa907/users/36bee95cf1dc4a98bce595fc5b0dee95/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:28.539087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1487125-5466-498f-8d98-7922c94c21b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:28.539710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1487125-5466-498f-8d98-7922c94c21b6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:28.540002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1487125-5466-498f-8d98-7922c94c21b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:28.540259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1487125-5466-498f-8d98-7922c94c21b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:28.540994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d844a-5a25-4bb7-bcb8-a5f6e5097f78 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:28.544259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1487125-5466-498f-8d98-7922c94c21b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:28.544622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1487125-5466-498f-8d98-7922c94c21b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:28.546085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1487125-5466-498f-8d98-7922c94c21b6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:28.564914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 128/261] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:28.572366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1487125-5466-498f-8d98-7922c94c21b6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:28.782163 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1487125-5466-498f-8d98-7922c94c21b6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:28.823317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d844a-5a25-4bb7-bcb8-a5f6e5097f78 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:28.841730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:28.850449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f9d9d-6b15-429b-a8fe-6001c0630b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:28.851106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f9d9d-6b15-429b-a8fe-6001c0630b04 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:28.851385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f9d9d-6b15-429b-a8fe-6001c0630b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:28.851672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f9d9d-6b15-429b-a8fe-6001c0630b04 None admin] PATH_INFO: `/v3/users/bb499eddbe004870aa0946730a98721e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:28.852917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f9d9d-6b15-429b-a8fe-6001c0630b04 None admin] RBAC: Authorizing `identity:get_user(user_id=bb499eddbe004870aa0946730a98721e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:28.854687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f9d9d-6b15-429b-a8fe-6001c0630b04 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:28.861231 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 129/262] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:28 2022] GET /identity/v3/users/bb499eddbe004870aa0946730a98721e => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:28.866375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f9d9d-6b15-429b-a8fe-6001c0630b04 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:28.876088 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:28.884665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f57833-9678-457a-b3d9-e62e012df765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:28.886819 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f57833-9678-457a-b3d9-e62e012df765 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:28.887321 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f57833-9678-457a-b3d9-e62e012df765 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:28.887822 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f57833-9678-457a-b3d9-e62e012df765 None admin] PATH_INFO: `/v3/users/bb499eddbe004870aa0946730a98721e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:28.889290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f57833-9678-457a-b3d9-e62e012df765 None admin] RBAC: Authorizing `identity:get_user(user_id=bb499eddbe004870aa0946730a98721e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:28.891032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f57833-9678-457a-b3d9-e62e012df765 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:28.892956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 134/263] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:28 2022] GET /identity/v3/users/bb499eddbe004870aa0946730a98721e => generated 283 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:28.899390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f57833-9678-457a-b3d9-e62e012df765 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:29.042288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f9d9d-6b15-429b-a8fe-6001c0630b04 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:29.093159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e85a2-b9c4-40d6-aeac-c83509bd05d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:29.094143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e85a2-b9c4-40d6-aeac-c83509bd05d4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:29.094777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e85a2-b9c4-40d6-aeac-c83509bd05d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:29.095255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e85a2-b9c4-40d6-aeac-c83509bd05d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:29.119796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f57833-9678-457a-b3d9-e62e012df765 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:29.184864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b05aac-f6a9-46da-bcde-546dfb396527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:29.185479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b05aac-f6a9-46da-bcde-546dfb396527 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:29.185753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b05aac-f6a9-46da-bcde-546dfb396527 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:29.186040 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b05aac-f6a9-46da-bcde-546dfb396527 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:29.213357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:29.218334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e85a2-b9c4-40d6-aeac-c83509bd05d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:29.220202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e85a2-b9c4-40d6-aeac-c83509bd05d4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:29.346709 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:29.349541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b05aac-f6a9-46da-bcde-546dfb396527 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:29.351784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b05aac-f6a9-46da-bcde-546dfb396527 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:29.397784 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 130/264] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 16:18:29.400259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0db1e-5a64-4db8-a7c8-ae6303d480e2 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:29.400549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0db1e-5a64-4db8-a7c8-ae6303d480e2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:29.400840 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0db1e-5a64-4db8-a7c8-ae6303d480e2 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:29.403559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 131/265] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:29.404483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0db1e-5a64-4db8-a7c8-ae6303d480e2 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:29.541142 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0db1e-5a64-4db8-a7c8-ae6303d480e2 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:29.543960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:29.560256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 135/266] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 16:18:29.564185 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562c964-7038-4d68-a46d-e3c51b51b294 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:29.564472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562c964-7038-4d68-a46d-e3c51b51b294 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:29.564744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562c964-7038-4d68-a46d-e3c51b51b294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:29.605202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbeffe0-0a05-41e1-9cb9-068544d61293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:29.605759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbeffe0-0a05-41e1-9cb9-068544d61293 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:29.606035 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbeffe0-0a05-41e1-9cb9-068544d61293 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:29.606305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbeffe0-0a05-41e1-9cb9-068544d61293 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:29.607409 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbeffe0-0a05-41e1-9cb9-068544d61293 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:29.608885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbeffe0-0a05-41e1-9cb9-068544d61293 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:29.609754 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-a562c964-7038-4d68-a46d-e3c51b51b294 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:29.610496 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 132/267] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:29 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:29.611255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a562c964-7038-4d68-a46d-e3c51b51b294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:29.611380 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbeffe0-0a05-41e1-9cb9-068544d61293 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:29.708849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 136/268] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:29.709775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562c964-7038-4d68-a46d-e3c51b51b294 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:29.765116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:29.775004 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbeffe0-0a05-41e1-9cb9-068544d61293 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:29.847990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12f8a9-ffec-4af1-9a4e-521b92b83f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:29.852458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12f8a9-ffec-4af1-9a4e-521b92b83f42 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:29.852949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12f8a9-ffec-4af1-9a4e-521b92b83f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:29.853632 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12f8a9-ffec-4af1-9a4e-521b92b83f42 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:29.855048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12f8a9-ffec-4af1-9a4e-521b92b83f42 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:29.856695 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12f8a9-ffec-4af1-9a4e-521b92b83f42 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:29.861938 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 133/269] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:29 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:29.863322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c1298-6022-4304-bed2-5558b6963912 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:29.863947 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c1298-6022-4304-bed2-5558b6963912 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:29.865075 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c1298-6022-4304-bed2-5558b6963912 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:29.869957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 134/270] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:29.870872 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c1298-6022-4304-bed2-5558b6963912 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:29.918942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562c964-7038-4d68-a46d-e3c51b51b294 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:29.920515 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:29.979790 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098db939-1c69-4671-806a-976a994a9382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:29.980436 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098db939-1c69-4671-806a-976a994a9382 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:29.980719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098db939-1c69-4671-806a-976a994a9382 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:29.981064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098db939-1c69-4671-806a-976a994a9382 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:29.981797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098db939-1c69-4671-806a-976a994a9382 None admin] RBAC: Adding query filter params (role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5, scope.project.id=cbda193ce0194e0294b307a9296fa907, user.id=bb499eddbe004870aa0946730a98721e) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:29.982159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098db939-1c69-4671-806a-976a994a9382 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:29.984228 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098db939-1c69-4671-806a-976a994a9382 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:29.990602 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 137/271] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:29 2022] GET /identity/v3/role_assignments?user.id=bb499eddbe004870aa0946730a98721e&scope.project.id=cbda193ce0194e0294b307a9296fa907&role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:29.996774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce554b-afd3-41dd-8123-44a98e1473bf None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:29.996774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce554b-afd3-41dd-8123-44a98e1473bf None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:29.996774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce554b-afd3-41dd-8123-44a98e1473bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:30.015584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-16ce554b-afd3-41dd-8123-44a98e1473bf None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:30.016882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-16ce554b-afd3-41dd-8123-44a98e1473bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:30.017709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c1298-6022-4304-bed2-5558b6963912 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:30.075991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873ebc2-d97e-44d8-abfc-808eb20680be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:30.078308 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873ebc2-d97e-44d8-abfc-808eb20680be None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:30.079947 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873ebc2-d97e-44d8-abfc-808eb20680be None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:30.080438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873ebc2-d97e-44d8-abfc-808eb20680be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:30.084688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873ebc2-d97e-44d8-abfc-808eb20680be None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=cbda193ce0194e0294b307a9296fa907, user.id=bb499eddbe004870aa0946730a98721e) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:30.085352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873ebc2-d97e-44d8-abfc-808eb20680be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:30.087127 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873ebc2-d97e-44d8-abfc-808eb20680be None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:30.111193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 135/272] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:29 2022] GET /identity/v3/role_assignments?user.id=bb499eddbe004870aa0946730a98721e&scope.project.id=cbda193ce0194e0294b307a9296fa907&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:30.112000 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873ebc2-d97e-44d8-abfc-808eb20680be None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:30.125754 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 138/273] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:30.235002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a49efc-57b6-4763-887f-acfec142ea8c None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:30.235347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a49efc-57b6-4763-887f-acfec142ea8c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:30.235562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a49efc-57b6-4763-887f-acfec142ea8c None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:30.237269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 139/274] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:18:30.243586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4e6dd-39b2-42b4-9d07-e45dc5f05bcb None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:30.243874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4e6dd-39b2-42b4-9d07-e45dc5f05bcb None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:30.244155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4e6dd-39b2-42b4-9d07-e45dc5f05bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:30.265450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-40d4e6dd-39b2-42b4-9d07-e45dc5f05bcb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:30.266673 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-40d4e6dd-39b2-42b4-9d07-e45dc5f05bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:30.288605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:30.292060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873ebc2-d97e-44d8-abfc-808eb20680be None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:30.326184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:30.362239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad4c7e-0ae9-430f-abd7-03a646605615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:30.362813 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad4c7e-0ae9-430f-abd7-03a646605615 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:30.363089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad4c7e-0ae9-430f-abd7-03a646605615 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:30.363357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad4c7e-0ae9-430f-abd7-03a646605615 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:30.379917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad4c7e-0ae9-430f-abd7-03a646605615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:30.385910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 140/275] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-73ad4c7e-0ae9-430f-abd7-03a646605615 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:30.386836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:30.387654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:30.389062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 136/276] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:30.394393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4e6dd-39b2-42b4-9d07-e45dc5f05bcb None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:30.468555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad4c7e-0ae9-430f-abd7-03a646605615 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:30.514671 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4e6dd-39b2-42b4-9d07-e45dc5f05bcb None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:30.562256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacfa37-6f47-4d52-a892-76d2923a6da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:30.562870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacfa37-6f47-4d52-a892-76d2923a6da0 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:30.563148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacfa37-6f47-4d52-a892-76d2923a6da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:30.563418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacfa37-6f47-4d52-a892-76d2923a6da0 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:30.564086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacfa37-6f47-4d52-a892-76d2923a6da0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:30.564383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacfa37-6f47-4d52-a892-76d2923a6da0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:30.565860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacfa37-6f47-4d52-a892-76d2923a6da0 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:30.581071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 141/277] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:30.587298 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacfa37-6f47-4d52-a892-76d2923a6da0 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:30.604846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad4c7e-0ae9-430f-abd7-03a646605615 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:30.614336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:30.663628 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfe66a-05e9-4335-89da-144f457db7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:30.664544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfe66a-05e9-4335-89da-144f457db7ef None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:30.665105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfe66a-05e9-4335-89da-144f457db7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:30.665571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfe66a-05e9-4335-89da-144f457db7ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:30.666869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfe66a-05e9-4335-89da-144f457db7ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:30.668516 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfe66a-05e9-4335-89da-144f457db7ef None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:30.674943 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 137/278] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:30.677142 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfe66a-05e9-4335-89da-144f457db7ef None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:30.714548 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:30.722906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacfa37-6f47-4d52-a892-76d2923a6da0 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:30.757375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85f214-a306-41be-ba9d-b0cc21ae0386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:30.758007 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85f214-a306-41be-ba9d-b0cc21ae0386 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:30.758280 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85f214-a306-41be-ba9d-b0cc21ae0386 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:30.758554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85f214-a306-41be-ba9d-b0cc21ae0386 None admin] PATH_INFO: `/v3/users/7e0884f6d1fe4a578676e8bc298fb7e6` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:30.771546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85f214-a306-41be-ba9d-b0cc21ae0386 None admin] RBAC: Authorizing `identity:get_user(user_id=7e0884f6d1fe4a578676e8bc298fb7e6)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:30.773185 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85f214-a306-41be-ba9d-b0cc21ae0386 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:30.774882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 142/279] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:30 2022] GET /identity/v3/users/7e0884f6d1fe4a578676e8bc298fb7e6 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:30.775642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85f214-a306-41be-ba9d-b0cc21ae0386 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:30.799593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfe66a-05e9-4335-89da-144f457db7ef None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:30.836014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fcbec-dc9d-4195-94f1-a8a60d9f7ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:30.836621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fcbec-dc9d-4195-94f1-a8a60d9f7ef1 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:30.836950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fcbec-dc9d-4195-94f1-a8a60d9f7ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:30.837206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fcbec-dc9d-4195-94f1-a8a60d9f7ef1 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:30.837855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fcbec-dc9d-4195-94f1-a8a60d9f7ef1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:30.839306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fcbec-dc9d-4195-94f1-a8a60d9f7ef1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:30.857198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:30.863603 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 138/280] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:18:30 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:18:30.864488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fcbec-dc9d-4195-94f1-a8a60d9f7ef1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:30.904855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85f214-a306-41be-ba9d-b0cc21ae0386 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:30.951694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04453f16-3cdb-44ac-bc4a-f5fd8dcaea2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:30.953286 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04453f16-3cdb-44ac-bc4a-f5fd8dcaea2f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:30.953858 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04453f16-3cdb-44ac-bc4a-f5fd8dcaea2f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:30.954324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04453f16-3cdb-44ac-bc4a-f5fd8dcaea2f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:30.967394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04453f16-3cdb-44ac-bc4a-f5fd8dcaea2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-04453f16-3cdb-44ac-bc4a-f5fd8dcaea2f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:30.973056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:30.973996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:30.974779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:30.974779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:30.974779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:30.974779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:30.974779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:30.974779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:30.974779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:30.974779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:30.974779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:30.974779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:30.974779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:30.974779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:30.981356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:30.984084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 143/281] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:30.985170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04453f16-3cdb-44ac-bc4a-f5fd8dcaea2f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:31.004401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fcbec-dc9d-4195-94f1-a8a60d9f7ef1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:31.060654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df809799-287b-49c1-88ff-26fc7be6f85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:31.061653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df809799-287b-49c1-88ff-26fc7be6f85a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:31.062002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df809799-287b-49c1-88ff-26fc7be6f85a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:31.062276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df809799-287b-49c1-88ff-26fc7be6f85a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:31.063093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df809799-287b-49c1-88ff-26fc7be6f85a None admin] RBAC: Adding query filter params (role.id=ac5a835695db44918d768db1bbb97589, user.id=7e0884f6d1fe4a578676e8bc298fb7e6, scope.system=all) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:31.063522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df809799-287b-49c1-88ff-26fc7be6f85a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:31.065104 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df809799-287b-49c1-88ff-26fc7be6f85a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:31.071287 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 139/282] 10.222.0.55 () {56 vars in 1308 bytes} [Fri Aug 19 16:18:30 2022] GET /identity/v3/role_assignments?user.id=7e0884f6d1fe4a578676e8bc298fb7e6&scope.system=all&role.id=ac5a835695db44918d768db1bbb97589 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:31.117139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04453f16-3cdb-44ac-bc4a-f5fd8dcaea2f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:31.121696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284c802-39d5-4b7d-8409-2b5461da9a88 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:31.121696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284c802-39d5-4b7d-8409-2b5461da9a88 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:31.121696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284c802-39d5-4b7d-8409-2b5461da9a88 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:31.123412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 140/283] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:18:31.130731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52baebf0-759b-4d96-aafa-2279ef94f608 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:31.130731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52baebf0-759b-4d96-aafa-2279ef94f608 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:31.130731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52baebf0-759b-4d96-aafa-2279ef94f608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:31.152383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-52baebf0-759b-4d96-aafa-2279ef94f608 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:31.154334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-52baebf0-759b-4d96-aafa-2279ef94f608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:31.164346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188491c8-9a58-40dc-9da2-655e427aa164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:31.165169 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188491c8-9a58-40dc-9da2-655e427aa164 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:31.165463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188491c8-9a58-40dc-9da2-655e427aa164 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:31.165815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188491c8-9a58-40dc-9da2-655e427aa164 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:31.166576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188491c8-9a58-40dc-9da2-655e427aa164 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:31.166955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188491c8-9a58-40dc-9da2-655e427aa164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:31.168478 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188491c8-9a58-40dc-9da2-655e427aa164 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:31.170266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:31.181342 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 144/284] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:31.187025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188491c8-9a58-40dc-9da2-655e427aa164 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:31.251637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 141/285] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:31.310056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188491c8-9a58-40dc-9da2-655e427aa164 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:31.346245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc1300-bb48-47bb-b7ec-3e84ee9d27e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:31.346888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc1300-bb48-47bb-b7ec-3e84ee9d27e2 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:31.347165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc1300-bb48-47bb-b7ec-3e84ee9d27e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:31.347444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc1300-bb48-47bb-b7ec-3e84ee9d27e2 None admin] PATH_INFO: `/v3/users/179267ebd4f143df9c0ef11de3c906be` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:31.348587 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc1300-bb48-47bb-b7ec-3e84ee9d27e2 None admin] RBAC: Authorizing `identity:get_user(user_id=179267ebd4f143df9c0ef11de3c906be)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:31.350074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc1300-bb48-47bb-b7ec-3e84ee9d27e2 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:31.352700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 145/286] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:31 2022] GET /identity/v3/users/179267ebd4f143df9c0ef11de3c906be => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:31.361869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52baebf0-759b-4d96-aafa-2279ef94f608 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:31.460679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:31.491226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52baebf0-759b-4d96-aafa-2279ef94f608 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:31.537305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff059833-592e-4432-9bdc-f2a04ba87e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:31.538600 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff059833-592e-4432-9bdc-f2a04ba87e39 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:31.538949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff059833-592e-4432-9bdc-f2a04ba87e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:31.539289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff059833-592e-4432-9bdc-f2a04ba87e39 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:31.540500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff059833-592e-4432-9bdc-f2a04ba87e39 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:31.542043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff059833-592e-4432-9bdc-f2a04ba87e39 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:31.544063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 142/287] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:31 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:31.549808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc1300-bb48-47bb-b7ec-3e84ee9d27e2 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:31.622484 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:31.624128 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff059833-592e-4432-9bdc-f2a04ba87e39 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:31.682037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc1300-bb48-47bb-b7ec-3e84ee9d27e2 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:31.769460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712b176-5348-4a2a-84a4-3099740161e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:31.770136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712b176-5348-4a2a-84a4-3099740161e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:31.770417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712b176-5348-4a2a-84a4-3099740161e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:31.770694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712b176-5348-4a2a-84a4-3099740161e2 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907/users/179267ebd4f143df9c0ef11de3c906be/roles/732934b017114352b11d54cb795862b1` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:31.773012 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff059833-592e-4432-9bdc-f2a04ba87e39 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:31.773247 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712b176-5348-4a2a-84a4-3099740161e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbda193ce0194e0294b307a9296fa907, user_id=179267ebd4f143df9c0ef11de3c906be, role_id=732934b017114352b11d54cb795862b1)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:31.774842 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712b176-5348-4a2a-84a4-3099740161e2 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:31.807205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 146/288] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:18:31 2022] PUT /identity/v3/projects/cbda193ce0194e0294b307a9296fa907/users/179267ebd4f143df9c0ef11de3c906be/roles/732934b017114352b11d54cb795862b1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:31.814824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1aff3-b427-4e3e-970f-ba8a518314a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:31.815519 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1aff3-b427-4e3e-970f-ba8a518314a1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:31.815861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1aff3-b427-4e3e-970f-ba8a518314a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:31.816124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1aff3-b427-4e3e-970f-ba8a518314a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:31.837860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1aff3-b427-4e3e-970f-ba8a518314a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-eed1aff3-b427-4e3e-970f-ba8a518314a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:31.843071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:31.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:31.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 143/289] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:31.852626 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712b176-5348-4a2a-84a4-3099740161e2 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:31.982920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712b176-5348-4a2a-84a4-3099740161e2 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:32.008285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:32.020240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a5835-4fee-46c9-a7f2-920584f25184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:32.020824 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a5835-4fee-46c9-a7f2-920584f25184 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:32.021111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a5835-4fee-46c9-a7f2-920584f25184 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:32.021563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a5835-4fee-46c9-a7f2-920584f25184 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:32.022528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a5835-4fee-46c9-a7f2-920584f25184 None admin] RBAC: Adding query filter params (name=member) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:32.022884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a5835-4fee-46c9-a7f2-920584f25184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:32.024675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a5835-4fee-46c9-a7f2-920584f25184 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:32.034815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 147/290] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:32.040553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1aff3-b427-4e3e-970f-ba8a518314a1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:32.053989 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:32.172046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1aff3-b427-4e3e-970f-ba8a518314a1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:32.222651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce4135-2745-4935-824d-eca8967dd3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:32.223538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce4135-2745-4935-824d-eca8967dd3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:32.224058 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce4135-2745-4935-824d-eca8967dd3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:32.224502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce4135-2745-4935-824d-eca8967dd3f8 None admin] PATH_INFO: `/v3/users/36bee95cf1dc4a98bce595fc5b0dee95` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:32.225940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce4135-2745-4935-824d-eca8967dd3f8 None admin] RBAC: Authorizing `identity:get_user(user_id=36bee95cf1dc4a98bce595fc5b0dee95)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:32.227542 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce4135-2745-4935-824d-eca8967dd3f8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:32.229575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 144/291] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:32 2022] GET /identity/v3/users/36bee95cf1dc4a98bce595fc5b0dee95 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:32.235406 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a5835-4fee-46c9-a7f2-920584f25184 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:32.359701 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a5835-4fee-46c9-a7f2-920584f25184 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:32.411414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf6a41-6087-4873-a5da-03b355df1a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:32.412023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf6a41-6087-4873-a5da-03b355df1a9f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:32.412301 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf6a41-6087-4873-a5da-03b355df1a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:32.412571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf6a41-6087-4873-a5da-03b355df1a9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:32.495396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:32.512421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf6a41-6087-4873-a5da-03b355df1a9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:32.514196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf6a41-6087-4873-a5da-03b355df1a9f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:32.648597 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 148/292] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:32.654622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce4135-2745-4935-824d-eca8967dd3f8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:32.754841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:32.795384 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce4135-2745-4935-824d-eca8967dd3f8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:32.835590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac65d52-dcae-4f54-b766-a30d58010ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:32.836235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac65d52-dcae-4f54-b766-a30d58010ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:32.836517 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac65d52-dcae-4f54-b766-a30d58010ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:32.836816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac65d52-dcae-4f54-b766-a30d58010ff7 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:32.838032 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac65d52-dcae-4f54-b766-a30d58010ff7 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:32.839473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac65d52-dcae-4f54-b766-a30d58010ff7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:32.841487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 145/293] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:32 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:32.849868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf6a41-6087-4873-a5da-03b355df1a9f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:32.955658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:32.976881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf6a41-6087-4873-a5da-03b355df1a9f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:33.016039 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02c1d7-0356-425a-b6a0-5ccef318494d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:33.016658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02c1d7-0356-425a-b6a0-5ccef318494d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:33.016936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02c1d7-0356-425a-b6a0-5ccef318494d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:33.017211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02c1d7-0356-425a-b6a0-5ccef318494d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:33.017933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02c1d7-0356-425a-b6a0-5ccef318494d None admin] RBAC: Adding query filter params (role.id=ac5a835695db44918d768db1bbb97589, scope.project.id=cbda193ce0194e0294b307a9296fa907, user.id=36bee95cf1dc4a98bce595fc5b0dee95) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:33.018238 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02c1d7-0356-425a-b6a0-5ccef318494d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:33.019684 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02c1d7-0356-425a-b6a0-5ccef318494d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:33.026553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 149/294] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:32 2022] GET /identity/v3/role_assignments?user.id=36bee95cf1dc4a98bce595fc5b0dee95&scope.project.id=cbda193ce0194e0294b307a9296fa907&role.id=ac5a835695db44918d768db1bbb97589 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:33.266075 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7643cca-0463-408c-9a03-38c5c44b9b3a None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:33.267150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7643cca-0463-408c-9a03-38c5c44b9b3a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:33.267690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7643cca-0463-408c-9a03-38c5c44b9b3a None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:33.269524 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 146/295] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:33.275199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1600655-7d76-4f66-b051-a58300a2a303 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:33.275490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1600655-7d76-4f66-b051-a58300a2a303 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:33.275764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1600655-7d76-4f66-b051-a58300a2a303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:33.296320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-a1600655-7d76-4f66-b051-a58300a2a303 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:33.297556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a1600655-7d76-4f66-b051-a58300a2a303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:33.372445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 150/296] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:33.534463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e4e66-f69a-4fe6-8eba-01f294b63f57 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:33.535063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e4e66-f69a-4fe6-8eba-01f294b63f57 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:33.535543 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e4e66-f69a-4fe6-8eba-01f294b63f57 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:33.537332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 147/297] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:33.543299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6e54a-349c-4687-b19f-de7305347756 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:33.543588 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6e54a-349c-4687-b19f-de7305347756 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:33.543859 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6e54a-349c-4687-b19f-de7305347756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:33.564185 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-24d6e54a-349c-4687-b19f-de7305347756 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:33.565384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-24d6e54a-349c-4687-b19f-de7305347756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:33.646391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 151/298] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:33.669227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e4e66-f69a-4fe6-8eba-01f294b63f57 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:33.684309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789fa80-e21f-4edc-9692-fb926f2c8e85 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:33.684618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789fa80-e21f-4edc-9692-fb926f2c8e85 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:33.684892 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789fa80-e21f-4edc-9692-fb926f2c8e85 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:33.686601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 152/299] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:18:33.691577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb540fe4-6cc5-4e0e-9a82-f2c06d4a5f89 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:33.691854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb540fe4-6cc5-4e0e-9a82-f2c06d4a5f89 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:33.692126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb540fe4-6cc5-4e0e-9a82-f2c06d4a5f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:33.710870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-eb540fe4-6cc5-4e0e-9a82-f2c06d4a5f89 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:33.712052 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-eb540fe4-6cc5-4e0e-9a82-f2c06d4a5f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:33.726345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:33.782690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 153/300] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:33.793872 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e4e66-f69a-4fe6-8eba-01f294b63f57 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:33.829018 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51faec6-6335-4e2f-8f6b-64cc1e2a488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:33.829653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51faec6-6335-4e2f-8f6b-64cc1e2a488c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:33.829931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51faec6-6335-4e2f-8f6b-64cc1e2a488c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:33.830206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51faec6-6335-4e2f-8f6b-64cc1e2a488c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:33.841202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51faec6-6335-4e2f-8f6b-64cc1e2a488c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-e51faec6-6335-4e2f-8f6b-64cc1e2a488c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:33.845936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:33.846785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:33.847544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 148/301] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:33.852940 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb540fe4-6cc5-4e0e-9a82-f2c06d4a5f89 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:33.956614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51faec6-6335-4e2f-8f6b-64cc1e2a488c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:33.958868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb540fe4-6cc5-4e0e-9a82-f2c06d4a5f89 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:33.993123 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:34.003916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c4e7b-4ec8-45ff-b92e-5c60d7439847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:34.004579 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c4e7b-4ec8-45ff-b92e-5c60d7439847 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:34.004893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c4e7b-4ec8-45ff-b92e-5c60d7439847 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:34.005170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c4e7b-4ec8-45ff-b92e-5c60d7439847 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:34.006199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c4e7b-4ec8-45ff-b92e-5c60d7439847 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:34.006702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c4e7b-4ec8-45ff-b92e-5c60d7439847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:34.007979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c4e7b-4ec8-45ff-b92e-5c60d7439847 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:34.019103 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 154/302] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:34.024891 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c4e7b-4ec8-45ff-b92e-5c60d7439847 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:34.027698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:34.078140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51faec6-6335-4e2f-8f6b-64cc1e2a488c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:34.112608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490da3e2-1896-421c-842a-ab702d75d2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:34.113263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490da3e2-1896-421c-842a-ab702d75d2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:34.113551 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490da3e2-1896-421c-842a-ab702d75d2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:34.113827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490da3e2-1896-421c-842a-ab702d75d2f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:34.124143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490da3e2-1896-421c-842a-ab702d75d2f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-490da3e2-1896-421c-842a-ab702d75d2f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:34.129041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:34.129863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:34.130866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 149/303] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:18:34.136428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490da3e2-1896-421c-842a-ab702d75d2f8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:34.137343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c4e7b-4ec8-45ff-b92e-5c60d7439847 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:34.173797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2015f-2c80-4373-a7aa-6abcc7bc1a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:34.174427 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2015f-2c80-4373-a7aa-6abcc7bc1a99 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:34.174702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2015f-2c80-4373-a7aa-6abcc7bc1a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:34.174968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2015f-2c80-4373-a7aa-6abcc7bc1a99 None admin] PATH_INFO: `/v3/users/2fee5cbb66c44a4a874cac6383313690` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:34.188689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2015f-2c80-4373-a7aa-6abcc7bc1a99 None admin] RBAC: Authorizing `identity:get_user(user_id=2fee5cbb66c44a4a874cac6383313690)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:34.190185 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2015f-2c80-4373-a7aa-6abcc7bc1a99 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:34.191812 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 155/304] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:34 2022] GET /identity/v3/users/2fee5cbb66c44a4a874cac6383313690 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:34.192654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2015f-2c80-4373-a7aa-6abcc7bc1a99 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:34.206793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:34.246923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490da3e2-1896-421c-842a-ab702d75d2f8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:34.250120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:34.282422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e635b1-86bd-4160-b1a1-3447d9f467c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:34.283057 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e635b1-86bd-4160-b1a1-3447d9f467c2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:34.283340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e635b1-86bd-4160-b1a1-3447d9f467c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:34.283614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e635b1-86bd-4160-b1a1-3447d9f467c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:34.284725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e635b1-86bd-4160-b1a1-3447d9f467c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:34.286588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e635b1-86bd-4160-b1a1-3447d9f467c2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:34.288334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 150/305] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:34.289493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e6f8d-dfae-4cf9-9b5b-60cf2364cfd1 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:34.289781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e6f8d-dfae-4cf9-9b5b-60cf2364cfd1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:34.290057 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e6f8d-dfae-4cf9-9b5b-60cf2364cfd1 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:34.291249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 151/306] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:34.291947 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e6f8d-dfae-4cf9-9b5b-60cf2364cfd1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:34.295997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2015f-2c80-4373-a7aa-6abcc7bc1a99 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:34.330880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ee5f0-b0e4-4536-8933-a029b4a09f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:34.331400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee5f0-b0e4-4536-8933-a029b4a09f9d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:34.331673 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee5f0-b0e4-4536-8933-a029b4a09f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:34.331941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee5f0-b0e4-4536-8933-a029b4a09f9d None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:34.332590 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ee5f0-b0e4-4536-8933-a029b4a09f9d None admin] RBAC: Adding query filter params (name=member) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:34.332913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ee5f0-b0e4-4536-8933-a029b4a09f9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:34.334279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ee5f0-b0e4-4536-8933-a029b4a09f9d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:34.342152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 156/307] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:34.342901 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ee5f0-b0e4-4536-8933-a029b4a09f9d None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:34.394568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e6f8d-dfae-4cf9-9b5b-60cf2364cfd1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:34.423632 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:34.430373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08a31a-cbee-4c56-aaae-71f83157605c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:34.430978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08a31a-cbee-4c56-aaae-71f83157605c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:34.431259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08a31a-cbee-4c56-aaae-71f83157605c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:34.431532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08a31a-cbee-4c56-aaae-71f83157605c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:34.432260 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08a31a-cbee-4c56-aaae-71f83157605c None admin] RBAC: Adding query filter params (role.id=732934b017114352b11d54cb795862b1, user.id=2fee5cbb66c44a4a874cac6383313690, scope.system=all) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:34.432564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08a31a-cbee-4c56-aaae-71f83157605c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:34.434029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08a31a-cbee-4c56-aaae-71f83157605c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:34.438981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 152/308] 10.222.0.55 () {56 vars in 1308 bytes} [Fri Aug 19 16:18:34 2022] GET /identity/v3/role_assignments?user.id=2fee5cbb66c44a4a874cac6383313690&scope.system=all&role.id=732934b017114352b11d54cb795862b1 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:34.441984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71551540-656c-4bdf-8965-af9bd2c57685 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:34.442264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71551540-656c-4bdf-8965-af9bd2c57685 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:34.442542 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71551540-656c-4bdf-8965-af9bd2c57685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:34.447753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ee5f0-b0e4-4536-8933-a029b4a09f9d None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:34.462786 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-71551540-656c-4bdf-8965-af9bd2c57685 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:34.464024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-71551540-656c-4bdf-8965-af9bd2c57685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:34.474228 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:34.487953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1ddfa-705e-4c6b-93c1-30c0d76cdd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:34.488596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ddfa-705e-4c6b-93c1-30c0d76cdd48 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:34.488926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ddfa-705e-4c6b-93c1-30c0d76cdd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:34.489203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ddfa-705e-4c6b-93c1-30c0d76cdd48 None admin] PATH_INFO: `/v3/groups` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:34.489857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1ddfa-705e-4c6b-93c1-30c0d76cdd48 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:34.492181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1ddfa-705e-4c6b-93c1-30c0d76cdd48 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:34.510761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 157/309] 10.222.0.55 () {60 vars in 1134 bytes} [Fri Aug 19 16:18:34 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:18:34.511676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1ddfa-705e-4c6b-93c1-30c0d76cdd48 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:34.560596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 153/310] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:34.658038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1ddfa-705e-4c6b-93c1-30c0d76cdd48 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:34.698384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f50aee-1d0c-42b9-a351-0708af4acf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:34.699293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f50aee-1d0c-42b9-a351-0708af4acf37 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:34.699569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f50aee-1d0c-42b9-a351-0708af4acf37 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:34.699838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f50aee-1d0c-42b9-a351-0708af4acf37 None admin] PATH_INFO: `/v3/users/7e0884f6d1fe4a578676e8bc298fb7e6` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:34.701001 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f50aee-1d0c-42b9-a351-0708af4acf37 None admin] RBAC: Authorizing `identity:get_user(user_id=7e0884f6d1fe4a578676e8bc298fb7e6)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:34.702435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f50aee-1d0c-42b9-a351-0708af4acf37 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:34.704172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 158/311] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:34 2022] GET /identity/v3/users/7e0884f6d1fe4a578676e8bc298fb7e6 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:34.709951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71551540-656c-4bdf-8965-af9bd2c57685 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:34.821469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71551540-656c-4bdf-8965-af9bd2c57685 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:34.831814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f50aee-1d0c-42b9-a351-0708af4acf37 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:34.847460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:34.863240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c816533-1906-4080-9032-9bc1aab5a8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:34.863900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c816533-1906-4080-9032-9bc1aab5a8d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:34.864176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c816533-1906-4080-9032-9bc1aab5a8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:34.864447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c816533-1906-4080-9032-9bc1aab5a8d1 None admin] PATH_INFO: `/v3/system/users/7e0884f6d1fe4a578676e8bc298fb7e6/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:34.866091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c816533-1906-4080-9032-9bc1aab5a8d1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7e0884f6d1fe4a578676e8bc298fb7e6, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:34.867363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c816533-1906-4080-9032-9bc1aab5a8d1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:34.884056 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 154/312] 10.222.0.55 () {58 vars in 1334 bytes} [Fri Aug 19 16:18:34 2022] PUT /identity/v3/system/users/7e0884f6d1fe4a578676e8bc298fb7e6/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:34.902023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:34.965027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f50aee-1d0c-42b9-a351-0708af4acf37 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:35.003738 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167390d2-bbfa-44fa-b608-73de1ebc5157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:35.004382 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167390d2-bbfa-44fa-b608-73de1ebc5157 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:35.004659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167390d2-bbfa-44fa-b608-73de1ebc5157 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:35.005001 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167390d2-bbfa-44fa-b608-73de1ebc5157 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:35.017568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167390d2-bbfa-44fa-b608-73de1ebc5157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-167390d2-bbfa-44fa-b608-73de1ebc5157 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:35.023086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:35.024461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:35.025265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 159/313] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:18:35.033220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c816533-1906-4080-9032-9bc1aab5a8d1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:35.145576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c816533-1906-4080-9032-9bc1aab5a8d1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:35.180654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9396dfb-785d-4eaa-8366-be395d7cfb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:35.181300 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9396dfb-785d-4eaa-8366-be395d7cfb12 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:35.181580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9396dfb-785d-4eaa-8366-be395d7cfb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:35.181856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9396dfb-785d-4eaa-8366-be395d7cfb12 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:35.182512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9396dfb-785d-4eaa-8366-be395d7cfb12 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:35.182812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9396dfb-785d-4eaa-8366-be395d7cfb12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:35.184180 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9396dfb-785d-4eaa-8366-be395d7cfb12 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:35.192546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 155/314] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:35.198024 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167390d2-bbfa-44fa-b608-73de1ebc5157 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:35.303082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167390d2-bbfa-44fa-b608-73de1ebc5157 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:35.314505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:35.339072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12e017-8ea0-482a-90a8-1f18acdc6cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:35.339698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12e017-8ea0-482a-90a8-1f18acdc6cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:35.339975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12e017-8ea0-482a-90a8-1f18acdc6cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:35.340245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12e017-8ea0-482a-90a8-1f18acdc6cb7 None admin] PATH_INFO: `/v3/users/179267ebd4f143df9c0ef11de3c906be` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:35.341371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a12e017-8ea0-482a-90a8-1f18acdc6cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=179267ebd4f143df9c0ef11de3c906be)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:35.342799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a12e017-8ea0-482a-90a8-1f18acdc6cb7 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:35.344540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 160/315] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:35 2022] GET /identity/v3/users/179267ebd4f143df9c0ef11de3c906be => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:35.350780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9396dfb-785d-4eaa-8366-be395d7cfb12 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:35.407915 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:35.462192 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9396dfb-785d-4eaa-8366-be395d7cfb12 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:35.498243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ccef8-3552-45e3-bbf0-50b2bef1abf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:35.499182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ccef8-3552-45e3-bbf0-50b2bef1abf6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:35.499636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ccef8-3552-45e3-bbf0-50b2bef1abf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:35.500091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ccef8-3552-45e3-bbf0-50b2bef1abf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:35.572062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ccef8-3552-45e3-bbf0-50b2bef1abf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:35.573848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ccef8-3552-45e3-bbf0-50b2bef1abf6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:35.628817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:35.688964 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 156/316] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:35.694422 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12e017-8ea0-482a-90a8-1f18acdc6cb7 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:35.798806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12e017-8ea0-482a-90a8-1f18acdc6cb7 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:35.833886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d4040-1d40-436f-bade-22f8aa3085d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:35.834452 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d4040-1d40-436f-bade-22f8aa3085d0 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:35.834857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d4040-1d40-436f-bade-22f8aa3085d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:35.835127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d4040-1d40-436f-bade-22f8aa3085d0 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:35.836245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d4040-1d40-436f-bade-22f8aa3085d0 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:35.837710 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d4040-1d40-436f-bade-22f8aa3085d0 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:35.839483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 161/317] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:35 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:35.845425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ccef8-3552-45e3-bbf0-50b2bef1abf6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:35.950886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ccef8-3552-45e3-bbf0-50b2bef1abf6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:35.985622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326f049-13a5-4f52-8795-8d46fbf26d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:35.986289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326f049-13a5-4f52-8795-8d46fbf26d48 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:35.986568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326f049-13a5-4f52-8795-8d46fbf26d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:35.986835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326f049-13a5-4f52-8795-8d46fbf26d48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:35.987556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326f049-13a5-4f52-8795-8d46fbf26d48 None admin] RBAC: Adding query filter params (role.id=732934b017114352b11d54cb795862b1, scope.project.id=cbda193ce0194e0294b307a9296fa907, user.id=179267ebd4f143df9c0ef11de3c906be) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:35.987857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326f049-13a5-4f52-8795-8d46fbf26d48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:35.989328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326f049-13a5-4f52-8795-8d46fbf26d48 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:36.000371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 157/318] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:18:35 2022] GET /identity/v3/role_assignments?user.id=179267ebd4f143df9c0ef11de3c906be&scope.project.id=cbda193ce0194e0294b307a9296fa907&role.id=732934b017114352b11d54cb795862b1 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:36.382909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c92504-c559-498b-8aa5-4c384945591c None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:36.383425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c92504-c559-498b-8aa5-4c384945591c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:36.383425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c92504-c559-498b-8aa5-4c384945591c None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:36.384965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 162/319] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:36.389862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2dab1-b7d9-435e-bfb7-2974374f8fc7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:36.390237 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2dab1-b7d9-435e-bfb7-2974374f8fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:36.390412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2dab1-b7d9-435e-bfb7-2974374f8fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:36.408350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-fdf2dab1-b7d9-435e-bfb7-2974374f8fc7 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:36.409583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fdf2dab1-b7d9-435e-bfb7-2974374f8fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:36.420158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:36.486479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 158/320] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:36.585334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34556e-37aa-41d1-870b-44f5d5ee6139 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:36.585642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34556e-37aa-41d1-870b-44f5d5ee6139 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:36.585886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34556e-37aa-41d1-870b-44f5d5ee6139 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:36.587233 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 163/321] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:36.592416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e94f87-d976-412c-8662-90323927aff3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:36.592709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e94f87-d976-412c-8662-90323927aff3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:36.593014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e94f87-d976-412c-8662-90323927aff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:36.611892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-38e94f87-d976-412c-8662-90323927aff3 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:36.613105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-38e94f87-d976-412c-8662-90323927aff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:36.688909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 159/322] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:36.751768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34556e-37aa-41d1-870b-44f5d5ee6139 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:36.803545 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:36.860095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34556e-37aa-41d1-870b-44f5d5ee6139 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:36.893192 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e157a7-76fe-4f0c-9456-b045a9a27f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:36.893821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e157a7-76fe-4f0c-9456-b045a9a27f2f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:36.894103 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e157a7-76fe-4f0c-9456-b045a9a27f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:36.894376 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e157a7-76fe-4f0c-9456-b045a9a27f2f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:36.905306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e157a7-76fe-4f0c-9456-b045a9a27f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-06e157a7-76fe-4f0c-9456-b045a9a27f2f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:36.910065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:36.910873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:36.911634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:36.911634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:36.911634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:36.911634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:36.911634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:36.911634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:36.911634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:36.911634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:36.911634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:36.911634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:36.911634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:36.911634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:36.913061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 164/323] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:36.914134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8c1d2-7cd2-42a2-b5b3-6e1782a9e6d0 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:36.914423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8c1d2-7cd2-42a2-b5b3-6e1782a9e6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:36.914698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8c1d2-7cd2-42a2-b5b3-6e1782a9e6d0 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:36.916034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 160/324] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:36.916888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e157a7-76fe-4f0c-9456-b045a9a27f2f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:36.921879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba03dbb-ef22-46e6-bafd-c426a86afbad None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:36.922161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba03dbb-ef22-46e6-bafd-c426a86afbad None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:36.922428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba03dbb-ef22-46e6-bafd-c426a86afbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:36.942066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5ba03dbb-ef22-46e6-bafd-c426a86afbad None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:36.943278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5ba03dbb-ef22-46e6-bafd-c426a86afbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:37.010472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 161/325] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:37.011311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba03dbb-ef22-46e6-bafd-c426a86afbad None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:37.014794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:37.020541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e157a7-76fe-4f0c-9456-b045a9a27f2f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:37.029184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:37.058462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5f288-918b-4453-a736-0e034f429876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:37.059072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5f288-918b-4453-a736-0e034f429876 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:37.059354 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5f288-918b-4453-a736-0e034f429876 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:37.059629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5f288-918b-4453-a736-0e034f429876 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:37.060294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5f288-918b-4453-a736-0e034f429876 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:37.060599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5f288-918b-4453-a736-0e034f429876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:37.062042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5f288-918b-4453-a736-0e034f429876 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:37.070974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 165/326] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:37.076535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5f288-918b-4453-a736-0e034f429876 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:37.129809 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba03dbb-ef22-46e6-bafd-c426a86afbad None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:37.166044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0a038-3541-43e4-bbbe-8f5ad4575920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:37.166601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0a038-3541-43e4-bbbe-8f5ad4575920 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:37.166878 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0a038-3541-43e4-bbbe-8f5ad4575920 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:37.167150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0a038-3541-43e4-bbbe-8f5ad4575920 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:37.168238 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0a038-3541-43e4-bbbe-8f5ad4575920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:37.169743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0a038-3541-43e4-bbbe-8f5ad4575920 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:37.171305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 162/327] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:37.176574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0a038-3541-43e4-bbbe-8f5ad4575920 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:37.183202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5f288-918b-4453-a736-0e034f429876 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:37.220304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75632c93-de7a-481c-8429-3ccac1651f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:37.220969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75632c93-de7a-481c-8429-3ccac1651f3c None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:37.221253 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75632c93-de7a-481c-8429-3ccac1651f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:37.221528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75632c93-de7a-481c-8429-3ccac1651f3c None admin] PATH_INFO: `/v3/users/2fee5cbb66c44a4a874cac6383313690` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:37.222652 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75632c93-de7a-481c-8429-3ccac1651f3c None admin] RBAC: Authorizing `identity:get_user(user_id=2fee5cbb66c44a4a874cac6383313690)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:37.224078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75632c93-de7a-481c-8429-3ccac1651f3c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:37.226147 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 166/328] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:37 2022] GET /identity/v3/users/2fee5cbb66c44a4a874cac6383313690 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:37.231996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75632c93-de7a-481c-8429-3ccac1651f3c None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:37.252954 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:37.271384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:37.284235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0a038-3541-43e4-bbbe-8f5ad4575920 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:37.317728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff326b-fb07-484f-a1a7-34b54b2f5800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:37.318334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff326b-fb07-484f-a1a7-34b54b2f5800 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:37.318608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff326b-fb07-484f-a1a7-34b54b2f5800 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:37.319436 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff326b-fb07-484f-a1a7-34b54b2f5800 None admin] PATH_INFO: `/v3/groups` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:37.320360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff326b-fb07-484f-a1a7-34b54b2f5800 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:37.321868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff326b-fb07-484f-a1a7-34b54b2f5800 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:37.340984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 163/329] 10.222.0.55 () {60 vars in 1134 bytes} [Fri Aug 19 16:18:37 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:18:37.340984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff326b-fb07-484f-a1a7-34b54b2f5800 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:37.341272 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75632c93-de7a-481c-8429-3ccac1651f3c None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:37.389746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c978eb-cfcf-480d-8bb3-d800fc3b40ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:37.391156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c978eb-cfcf-480d-8bb3-d800fc3b40ca None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:37.391445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c978eb-cfcf-480d-8bb3-d800fc3b40ca None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:37.392082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c978eb-cfcf-480d-8bb3-d800fc3b40ca None admin] PATH_INFO: `/v3/system/users/2fee5cbb66c44a4a874cac6383313690/roles/732934b017114352b11d54cb795862b1` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:37.393773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c978eb-cfcf-480d-8bb3-d800fc3b40ca None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2fee5cbb66c44a4a874cac6383313690, role_id=732934b017114352b11d54cb795862b1)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:37.395585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c978eb-cfcf-480d-8bb3-d800fc3b40ca None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:37.406075 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 167/330] 10.222.0.55 () {58 vars in 1334 bytes} [Fri Aug 19 16:18:37 2022] PUT /identity/v3/system/users/2fee5cbb66c44a4a874cac6383313690/roles/732934b017114352b11d54cb795862b1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:18:37.471743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff326b-fb07-484f-a1a7-34b54b2f5800 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:37.508515 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:37.516682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359874d-69f3-4a94-8529-c67a4470fc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:37.517537 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359874d-69f3-4a94-8529-c67a4470fc85 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:37.518026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359874d-69f3-4a94-8529-c67a4470fc85 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:37.518303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359874d-69f3-4a94-8529-c67a4470fc85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:37.530597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359874d-69f3-4a94-8529-c67a4470fc85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a359874d-69f3-4a94-8529-c67a4470fc85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:37.535939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:37.536904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:37.537761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 164/331] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:37.543490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c978eb-cfcf-480d-8bb3-d800fc3b40ca None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:37.660641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c978eb-cfcf-480d-8bb3-d800fc3b40ca None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:37.663679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:37.701345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e2d28-d386-424d-b0aa-2d8c0c269fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:37.701981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e2d28-d386-424d-b0aa-2d8c0c269fed None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:37.702263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e2d28-d386-424d-b0aa-2d8c0c269fed None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:37.702534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e2d28-d386-424d-b0aa-2d8c0c269fed None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:37.703199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e2d28-d386-424d-b0aa-2d8c0c269fed None admin] RBAC: Adding query filter params (name=member) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:37.703499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e2d28-d386-424d-b0aa-2d8c0c269fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:37.704955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e2d28-d386-424d-b0aa-2d8c0c269fed None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:37.714251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 168/332] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:37.719641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359874d-69f3-4a94-8529-c67a4470fc85 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:37.836898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359874d-69f3-4a94-8529-c67a4470fc85 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:37.878096 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c7b19-8cbc-434c-837d-20e4a13066e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:37.879001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c7b19-8cbc-434c-837d-20e4a13066e9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:37.879281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c7b19-8cbc-434c-837d-20e4a13066e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:37.879557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c7b19-8cbc-434c-837d-20e4a13066e9 None admin] PATH_INFO: `/v3/users/7e0884f6d1fe4a578676e8bc298fb7e6` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:37.880722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c7b19-8cbc-434c-837d-20e4a13066e9 None admin] RBAC: Authorizing `identity:get_user(user_id=7e0884f6d1fe4a578676e8bc298fb7e6)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:37.882256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c7b19-8cbc-434c-837d-20e4a13066e9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:37.884213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 165/333] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:37 2022] GET /identity/v3/users/7e0884f6d1fe4a578676e8bc298fb7e6 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:37.890188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e2d28-d386-424d-b0aa-2d8c0c269fed None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:38.000789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e2d28-d386-424d-b0aa-2d8c0c269fed None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:38.043805 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bacd2-66b0-4c36-9bbe-ba8733c3ce21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:38.044432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bacd2-66b0-4c36-9bbe-ba8733c3ce21 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:38.044713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bacd2-66b0-4c36-9bbe-ba8733c3ce21 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:38.045020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bacd2-66b0-4c36-9bbe-ba8733c3ce21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:38.045755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bacd2-66b0-4c36-9bbe-ba8733c3ce21 None admin] RBAC: Adding query filter params (role.id=ac5a835695db44918d768db1bbb97589, user.id=7e0884f6d1fe4a578676e8bc298fb7e6, scope.system=all) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:38.046065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bacd2-66b0-4c36-9bbe-ba8733c3ce21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:38.047521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bacd2-66b0-4c36-9bbe-ba8733c3ce21 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:38.061846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 169/334] 10.222.0.55 () {56 vars in 1308 bytes} [Fri Aug 19 16:18:37 2022] GET /identity/v3/role_assignments?user.id=7e0884f6d1fe4a578676e8bc298fb7e6&scope.system=all&role.id=ac5a835695db44918d768db1bbb97589 => generated 510 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:39.497853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd9dfd-e490-4ebd-bf86-9bb6c6a8ecd9 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:39.498361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd9dfd-e490-4ebd-bf86-9bb6c6a8ecd9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:39.498467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd9dfd-e490-4ebd-bf86-9bb6c6a8ecd9 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:39.500006 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 166/335] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:39.505282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077095a4-6a96-4ff6-ba4d-fb49c7390660 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:39.506118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077095a4-6a96-4ff6-ba4d-fb49c7390660 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:39.506619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077095a4-6a96-4ff6-ba4d-fb49c7390660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:39.527602 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-077095a4-6a96-4ff6-ba4d-fb49c7390660 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:39.529190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-077095a4-6a96-4ff6-ba4d-fb49c7390660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:39.554019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:39.617400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 170/336] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:39.774615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f51f81-9242-45f6-946e-7037f710f900 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:39.774802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f51f81-9242-45f6-946e-7037f710f900 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:39.775088 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f51f81-9242-45f6-946e-7037f710f900 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:39.776569 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 167/337] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:39.782186 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e896d7-11d5-4995-acd3-4e2b6788e47f None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:39.782703 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e896d7-11d5-4995-acd3-4e2b6788e47f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:39.783187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e896d7-11d5-4995-acd3-4e2b6788e47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:39.808927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-04e896d7-11d5-4995-acd3-4e2b6788e47f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:39.810456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-04e896d7-11d5-4995-acd3-4e2b6788e47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:39.893614 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 171/338] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:40.098876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dec8e3-e743-4285-9927-12a5351a29b3 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:40.099560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dec8e3-e743-4285-9927-12a5351a29b3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:40.100119 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dec8e3-e743-4285-9927-12a5351a29b3 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:40.102882 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 168/339] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:40.108481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f695a9e-714b-4011-8e19-68812c3b7a83 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:40.109056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f695a9e-714b-4011-8e19-68812c3b7a83 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:40.109554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f695a9e-714b-4011-8e19-68812c3b7a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:40.135228 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-8f695a9e-714b-4011-8e19-68812c3b7a83 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:40.137019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8f695a9e-714b-4011-8e19-68812c3b7a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:40.164677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dec8e3-e743-4285-9927-12a5351a29b3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:40.231429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 172/340] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:40.258235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:40.305385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dec8e3-e743-4285-9927-12a5351a29b3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:40.343858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870d870-f3d7-4744-96dd-f6a769aff6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:40.344502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870d870-f3d7-4744-96dd-f6a769aff6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:40.344803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870d870-f3d7-4744-96dd-f6a769aff6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:40.345082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870d870-f3d7-4744-96dd-f6a769aff6b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:40.356641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870d870-f3d7-4744-96dd-f6a769aff6b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c870d870-f3d7-4744-96dd-f6a769aff6b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:40.363581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:40.364509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:40.365311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 169/341] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:40.372085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f695a9e-714b-4011-8e19-68812c3b7a83 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:40.392177 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:40.426834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870d870-f3d7-4744-96dd-f6a769aff6b8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:40.522300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f695a9e-714b-4011-8e19-68812c3b7a83 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:40.582065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a424c-53a2-4bcc-9975-fca63ceb008b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:40.582700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a424c-53a2-4bcc-9975-fca63ceb008b None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:40.582984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a424c-53a2-4bcc-9975-fca63ceb008b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:40.583259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a424c-53a2-4bcc-9975-fca63ceb008b None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:40.583934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a424c-53a2-4bcc-9975-fca63ceb008b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:40.584242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a424c-53a2-4bcc-9975-fca63ceb008b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:40.585740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a424c-53a2-4bcc-9975-fca63ceb008b None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:40.588255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870d870-f3d7-4744-96dd-f6a769aff6b8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:40.604681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 173/342] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:40.610284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a424c-53a2-4bcc-9975-fca63ceb008b None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:40.624929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:40.671393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f494f-295a-431f-901d-79d531a65afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:40.672053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f494f-295a-431f-901d-79d531a65afe None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:40.672343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f494f-295a-431f-901d-79d531a65afe None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:40.672675 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f494f-295a-431f-901d-79d531a65afe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:40.713176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f494f-295a-431f-901d-79d531a65afe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-8b5f494f-295a-431f-901d-79d531a65afe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:40.718659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:40.719538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:40.720347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 170/343] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:40.726148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:40.733799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f494f-295a-431f-901d-79d531a65afe None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:40.764134 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a424c-53a2-4bcc-9975-fca63ceb008b None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:40.838105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72794057-db6b-49e0-b261-45383e48a5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:40.838772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72794057-db6b-49e0-b261-45383e48a5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:40.839051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72794057-db6b-49e0-b261-45383e48a5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:40.839329 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72794057-db6b-49e0-b261-45383e48a5e4 None admin] PATH_INFO: `/v3/users/2fee5cbb66c44a4a874cac6383313690` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:40.848183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72794057-db6b-49e0-b261-45383e48a5e4 None admin] RBAC: Authorizing `identity:get_user(user_id=2fee5cbb66c44a4a874cac6383313690)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:40.849703 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72794057-db6b-49e0-b261-45383e48a5e4 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:40.852213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 174/344] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:18:40 2022] GET /identity/v3/users/2fee5cbb66c44a4a874cac6383313690 => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:40.854778 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7367b29-08e2-48ca-8e6d-ebccd8208e4d None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:40.855074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7367b29-08e2-48ca-8e6d-ebccd8208e4d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:40.855350 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7367b29-08e2-48ca-8e6d-ebccd8208e4d None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:40.856715 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 175/345] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:40.866400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7367b29-08e2-48ca-8e6d-ebccd8208e4d None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:40.882943 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f494f-295a-431f-901d-79d531a65afe None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:40.923704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b838bd3-110b-4ef1-8b67-0bde60b85f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:40.924341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b838bd3-110b-4ef1-8b67-0bde60b85f73 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:40.924601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b838bd3-110b-4ef1-8b67-0bde60b85f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:40.924919 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b838bd3-110b-4ef1-8b67-0bde60b85f73 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:40.925597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b838bd3-110b-4ef1-8b67-0bde60b85f73 None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:40.925899 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b838bd3-110b-4ef1-8b67-0bde60b85f73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:40.927428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b838bd3-110b-4ef1-8b67-0bde60b85f73 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:40.938001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 171/346] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:40.939163 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dccaa-d257-4240-afad-3676d4db0661 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:40.939456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dccaa-d257-4240-afad-3676d4db0661 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:40.939733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dccaa-d257-4240-afad-3676d4db0661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:40.969447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3c0dccaa-d257-4240-afad-3676d4db0661 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:40.969447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3c0dccaa-d257-4240-afad-3676d4db0661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:41.014997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:41.033534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7367b29-08e2-48ca-8e6d-ebccd8208e4d None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:41.054793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 172/347] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:41.061637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dccaa-d257-4240-afad-3676d4db0661 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:41.098021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c66dc1-cc30-4c07-8595-7258eac502ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:41.098896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c66dc1-cc30-4c07-8595-7258eac502ab None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:41.099006 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c66dc1-cc30-4c07-8595-7258eac502ab None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:41.099379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c66dc1-cc30-4c07-8595-7258eac502ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:41.100718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c66dc1-cc30-4c07-8595-7258eac502ab None admin] RBAC: Adding query filter params (role.id=732934b017114352b11d54cb795862b1, user.id=2fee5cbb66c44a4a874cac6383313690, scope.system=all) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:41.102043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c66dc1-cc30-4c07-8595-7258eac502ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:41.102857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c66dc1-cc30-4c07-8595-7258eac502ab None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:41.138115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:41.163616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 176/348] 10.222.0.55 () {56 vars in 1308 bytes} [Fri Aug 19 16:18:40 2022] GET /identity/v3/role_assignments?user.id=2fee5cbb66c44a4a874cac6383313690&scope.system=all&role.id=732934b017114352b11d54cb795862b1 => generated 510 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:41.164486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c66dc1-cc30-4c07-8595-7258eac502ab None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:41.200099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dccaa-d257-4240-afad-3676d4db0661 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:41.244709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709873b9-e5ea-469a-b339-4994e0239027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:41.245408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709873b9-e5ea-469a-b339-4994e0239027 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:41.245688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709873b9-e5ea-469a-b339-4994e0239027 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:41.245963 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709873b9-e5ea-469a-b339-4994e0239027 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:41.275883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709873b9-e5ea-469a-b339-4994e0239027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-709873b9-e5ea-469a-b339-4994e0239027 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:41.283590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:41.284630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:41.286154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:41.286154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:41.286154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:41.286154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:41.286154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:41.286154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:41.286154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:41.286154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:41.286154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:41.286154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:41.286154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:41.286154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:41.286990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 173/349] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:41.292832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709873b9-e5ea-469a-b339-4994e0239027 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:41.349367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:41.404722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c66dc1-cc30-4c07-8595-7258eac502ab None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:41.430618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709873b9-e5ea-469a-b339-4994e0239027 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:41.446768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e470c-d4ee-4b37-86a1-1a0d4d5bc9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:41.447397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e470c-d4ee-4b37-86a1-1a0d4d5bc9bf None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:41.447679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e470c-d4ee-4b37-86a1-1a0d4d5bc9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:41.447952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e470c-d4ee-4b37-86a1-1a0d4d5bc9bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:41.475928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0395fcf-0312-45f0-a912-abedb507ee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:41.476657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0395fcf-0312-45f0-a912-abedb507ee20 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:41.477013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0395fcf-0312-45f0-a912-abedb507ee20 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:41.477293 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0395fcf-0312-45f0-a912-abedb507ee20 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:41.477967 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0395fcf-0312-45f0-a912-abedb507ee20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:41.478523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0395fcf-0312-45f0-a912-abedb507ee20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:41.480127 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0395fcf-0312-45f0-a912-abedb507ee20 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:41.489243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 174/350] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:41.490189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0395fcf-0312-45f0-a912-abedb507ee20 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:41.520676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:41.525278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e470c-d4ee-4b37-86a1-1a0d4d5bc9bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:41.526761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e470c-d4ee-4b37-86a1-1a0d4d5bc9bf None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:41.624222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:41.634173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0395fcf-0312-45f0-a912-abedb507ee20 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:41.669915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 177/351] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:41.672796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e470c-d4ee-4b37-86a1-1a0d4d5bc9bf None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:41.680252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83bcb1-0a14-44fb-8730-4ea1c204a708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:41.680941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83bcb1-0a14-44fb-8730-4ea1c204a708 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:41.681224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83bcb1-0a14-44fb-8730-4ea1c204a708 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:41.681495 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83bcb1-0a14-44fb-8730-4ea1c204a708 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:41.698784 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b83bcb1-0a14-44fb-8730-4ea1c204a708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-3b83bcb1-0a14-44fb-8730-4ea1c204a708 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:41.704044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:41.704961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 175/352] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:41.708980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13577d69-071d-4985-b1af-a217b883b1ec None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:41.709751 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13577d69-071d-4985-b1af-a217b883b1ec None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:41.709751 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13577d69-071d-4985-b1af-a217b883b1ec None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:41.709751 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 176/353] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:41.709751 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13577d69-071d-4985-b1af-a217b883b1ec None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:41.801260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:41.807618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e470c-d4ee-4b37-86a1-1a0d4d5bc9bf None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:41.844892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13577d69-071d-4985-b1af-a217b883b1ec None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:41.854642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46afe1-28b5-4d8c-b116-6292fc9b0374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:41.855618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46afe1-28b5-4d8c-b116-6292fc9b0374 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:41.855899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46afe1-28b5-4d8c-b116-6292fc9b0374 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:41.856171 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46afe1-28b5-4d8c-b116-6292fc9b0374 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:41.887489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d78b99-40cb-471b-8ee1-ccf4acaa3223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:41.888137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d78b99-40cb-471b-8ee1-ccf4acaa3223 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:41.888797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d78b99-40cb-471b-8ee1-ccf4acaa3223 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:41.888797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d78b99-40cb-471b-8ee1-ccf4acaa3223 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:41.889880 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d78b99-40cb-471b-8ee1-ccf4acaa3223 None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:41.891348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d78b99-40cb-471b-8ee1-ccf4acaa3223 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:41.893363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 177/354] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:41 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:41.894578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6dfee-16f9-4647-ab00-67b3cfe2abcc None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:41.894866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6dfee-16f9-4647-ab00-67b3cfe2abcc None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:41.895143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6dfee-16f9-4647-ab00-67b3cfe2abcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:41.913831 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-10f6dfee-16f9-4647-ab00-67b3cfe2abcc None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:41.914967 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-10f6dfee-16f9-4647-ab00-67b3cfe2abcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:41.933618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:41.958970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46afe1-28b5-4d8c-b116-6292fc9b0374 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:41.960465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46afe1-28b5-4d8c-b116-6292fc9b0374 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:42.004866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 178/355] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:42.005893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6dfee-16f9-4647-ab00-67b3cfe2abcc None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:42.092003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:42.096933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 178/356] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:42.098796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46afe1-28b5-4d8c-b116-6292fc9b0374 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:42.138086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6dfee-16f9-4647-ab00-67b3cfe2abcc None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:42.185593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5d66c-65ce-405f-a41a-634c4b6bcb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:42.186439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5d66c-65ce-405f-a41a-634c4b6bcb3e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:42.186752 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5d66c-65ce-405f-a41a-634c4b6bcb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:42.187670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5d66c-65ce-405f-a41a-634c4b6bcb3e None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:42.188300 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5d66c-65ce-405f-a41a-634c4b6bcb3e None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:42.188891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5d66c-65ce-405f-a41a-634c4b6bcb3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:42.190565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5d66c-65ce-405f-a41a-634c4b6bcb3e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:42.202087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 179/357] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:42.203324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb115a7-5703-422d-9826-20ce57f23644 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:42.203617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb115a7-5703-422d-9826-20ce57f23644 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:42.203894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb115a7-5703-422d-9826-20ce57f23644 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:42.205638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 180/358] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:42.206262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb115a7-5703-422d-9826-20ce57f23644 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:42.229176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:42.231077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46afe1-28b5-4d8c-b116-6292fc9b0374 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:42.274464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d76cd3-4374-44ce-a551-925cbaec39ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:42.275396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d76cd3-4374-44ce-a551-925cbaec39ed None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:42.275851 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d76cd3-4374-44ce-a551-925cbaec39ed None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:42.276263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d76cd3-4374-44ce-a551-925cbaec39ed None admin] PATH_INFO: `/v3/groups/0db9f5507e2a4efe94d8fc33470c310e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:42.282979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d76cd3-4374-44ce-a551-925cbaec39ed None admin] RBAC: Authorizing `identity:get_group(group_id=0db9f5507e2a4efe94d8fc33470c310e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:42.284789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d76cd3-4374-44ce-a551-925cbaec39ed None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:42.288038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 179/359] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:42 2022] GET /identity/v3/groups/0db9f5507e2a4efe94d8fc33470c310e => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:42.288993 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d76cd3-4374-44ce-a551-925cbaec39ed None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:42.339481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb115a7-5703-422d-9826-20ce57f23644 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:42.371817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:42.384710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c014636-d2a5-438f-8cdc-cc8907406b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:42.385427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014636-d2a5-438f-8cdc-cc8907406b61 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:42.385768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014636-d2a5-438f-8cdc-cc8907406b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:42.386047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014636-d2a5-438f-8cdc-cc8907406b61 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:42.387595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c014636-d2a5-438f-8cdc-cc8907406b61 None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:42.388684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c014636-d2a5-438f-8cdc-cc8907406b61 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:42.390377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 181/360] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:42 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:42.391595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6014c50-fcba-4b61-a6ed-dc5d47c0cd71 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:42.391988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6014c50-fcba-4b61-a6ed-dc5d47c0cd71 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:42.392275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6014c50-fcba-4b61-a6ed-dc5d47c0cd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:42.414159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d6014c50-fcba-4b61-a6ed-dc5d47c0cd71 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:42.415352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d76cd3-4374-44ce-a551-925cbaec39ed None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:42.415479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d6014c50-fcba-4b61-a6ed-dc5d47c0cd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:42.451806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a4336-0f82-437c-8bfd-a4915f16eb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:42.452336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a4336-0f82-437c-8bfd-a4915f16eb13 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:42.452613 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a4336-0f82-437c-8bfd-a4915f16eb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:42.452911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a4336-0f82-437c-8bfd-a4915f16eb13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:42.492332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 182/361] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:42.493314 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6014c50-fcba-4b61-a6ed-dc5d47c0cd71 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:42.510066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:42.533976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a4336-0f82-437c-8bfd-a4915f16eb13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:42.535528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a4336-0f82-437c-8bfd-a4915f16eb13 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:42.611065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:42.629795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6014c50-fcba-4b61-a6ed-dc5d47c0cd71 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:42.666891 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 180/362] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:42.672181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a4336-0f82-437c-8bfd-a4915f16eb13 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:42.686047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd94444-62e0-4c94-8044-fcae52fa2a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:42.686993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd94444-62e0-4c94-8044-fcae52fa2a29 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:42.687494 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd94444-62e0-4c94-8044-fcae52fa2a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:42.688173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd94444-62e0-4c94-8044-fcae52fa2a29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:42.690194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd94444-62e0-4c94-8044-fcae52fa2a29 None admin] RBAC: Adding query filter params (group.id=0db9f5507e2a4efe94d8fc33470c310e, role.id=ac5a835695db44918d768db1bbb97589, scope.project.id=9164929bc082468aa5cb93ca3d057091) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:42.690730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd94444-62e0-4c94-8044-fcae52fa2a29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:42.692690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd94444-62e0-4c94-8044-fcae52fa2a29 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:42.699599 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 183/363] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Aug 19 16:18:42 2022] GET /identity/v3/role_assignments?group.id=0db9f5507e2a4efe94d8fc33470c310e&scope.project.id=9164929bc082468aa5cb93ca3d057091&role.id=ac5a835695db44918d768db1bbb97589 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:42.700677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd94444-62e0-4c94-8044-fcae52fa2a29 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:42.804871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a4336-0f82-437c-8bfd-a4915f16eb13 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:42.805081 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:42.840401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd94444-62e0-4c94-8044-fcae52fa2a29 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:42.855104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08df4a-33cd-4b52-ab29-54728d2f8b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:42.856016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08df4a-33cd-4b52-ab29-54728d2f8b26 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:42.856315 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08df4a-33cd-4b52-ab29-54728d2f8b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:42.857127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08df4a-33cd-4b52-ab29-54728d2f8b26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:42.875065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08df4a-33cd-4b52-ab29-54728d2f8b26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-0c08df4a-33cd-4b52-ab29-54728d2f8b26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:42.879906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:42.880775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:42.882576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 181/364] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:42.883274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08df4a-33cd-4b52-ab29-54728d2f8b26 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:42.888359 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627cad8-a3bf-4efa-bdff-59ed2c9e32aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:42.888682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627cad8-a3bf-4efa-bdff-59ed2c9e32aa None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:42.888994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627cad8-a3bf-4efa-bdff-59ed2c9e32aa None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:42.889270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627cad8-a3bf-4efa-bdff-59ed2c9e32aa None admin] PATH_INFO: `/v3/groups/0db9f5507e2a4efe94d8fc33470c310e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:42.891212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627cad8-a3bf-4efa-bdff-59ed2c9e32aa None admin] RBAC: Authorizing `identity:get_group(group_id=0db9f5507e2a4efe94d8fc33470c310e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:42.892660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627cad8-a3bf-4efa-bdff-59ed2c9e32aa None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:42.894670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 184/365] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:42 2022] GET /identity/v3/groups/0db9f5507e2a4efe94d8fc33470c310e => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:42.895463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627cad8-a3bf-4efa-bdff-59ed2c9e32aa None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:42.910307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:43.010799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08df4a-33cd-4b52-ab29-54728d2f8b26 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:43.027831 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627cad8-a3bf-4efa-bdff-59ed2c9e32aa None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:43.048989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf258f9-9d06-447a-bfce-36d706e3f5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:43.049623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf258f9-9d06-447a-bfce-36d706e3f5dc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:43.049897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf258f9-9d06-447a-bfce-36d706e3f5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:43.050173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf258f9-9d06-447a-bfce-36d706e3f5dc None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:43.051302 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf258f9-9d06-447a-bfce-36d706e3f5dc None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:43.052775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf258f9-9d06-447a-bfce-36d706e3f5dc None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:43.054747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 182/366] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:42 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:43.055579 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf258f9-9d06-447a-bfce-36d706e3f5dc None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:43.068221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d413c8c-1b59-474c-ac99-37349e112794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:43.069799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d413c8c-1b59-474c-ac99-37349e112794 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:43.070278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d413c8c-1b59-474c-ac99-37349e112794 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:43.070767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d413c8c-1b59-474c-ac99-37349e112794 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:43.082791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d413c8c-1b59-474c-ac99-37349e112794 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4d413c8c-1b59-474c-ac99-37349e112794 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:43.089190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:43.090074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:43.090856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:43.090856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:43.090856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:43.090856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:43.090856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:43.090856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:43.090856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:43.090856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:43.090856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:43.090856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:43.090856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:43.090856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:43.093993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 185/367] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:18:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:43.094986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d413c8c-1b59-474c-ac99-37349e112794 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:43.104966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:43.165645 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:43.184332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf258f9-9d06-447a-bfce-36d706e3f5dc None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:43.227479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732238ff-2f86-415c-879e-a138a5d5aa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:43.228520 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732238ff-2f86-415c-879e-a138a5d5aa38 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:43.228893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732238ff-2f86-415c-879e-a138a5d5aa38 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:43.229150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732238ff-2f86-415c-879e-a138a5d5aa38 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:43.230383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732238ff-2f86-415c-879e-a138a5d5aa38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:43.230966 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732238ff-2f86-415c-879e-a138a5d5aa38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:43.235201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732238ff-2f86-415c-879e-a138a5d5aa38 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:43.235327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d413c8c-1b59-474c-ac99-37349e112794 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:43.243673 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 183/368] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:43.245069 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732238ff-2f86-415c-879e-a138a5d5aa38 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:43.275300 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f6972-fabe-41da-9da6-bdc9f538b6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:43.276109 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f6972-fabe-41da-9da6-bdc9f538b6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:43.276458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f6972-fabe-41da-9da6-bdc9f538b6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:43.276823 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f6972-fabe-41da-9da6-bdc9f538b6c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:43.277621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f6972-fabe-41da-9da6-bdc9f538b6c1 None admin] RBAC: Adding query filter params (group.id=0db9f5507e2a4efe94d8fc33470c310e, role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5, scope.project.id=9164929bc082468aa5cb93ca3d057091) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:43.277928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f6972-fabe-41da-9da6-bdc9f538b6c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:43.279455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f6972-fabe-41da-9da6-bdc9f538b6c1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:43.286602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 186/369] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Aug 19 16:18:43 2022] GET /identity/v3/role_assignments?group.id=0db9f5507e2a4efe94d8fc33470c310e&scope.project.id=9164929bc082468aa5cb93ca3d057091&role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:43.287483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f6972-fabe-41da-9da6-bdc9f538b6c1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:43.383906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:43.386019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732238ff-2f86-415c-879e-a138a5d5aa38 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:43.424593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48bfad-82e0-48ad-9bef-406fda1c4f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:43.425275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48bfad-82e0-48ad-9bef-406fda1c4f50 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:43.425612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48bfad-82e0-48ad-9bef-406fda1c4f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:43.425887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48bfad-82e0-48ad-9bef-406fda1c4f50 None admin] PATH_INFO: `/v3/groups/0db9f5507e2a4efe94d8fc33470c310e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:43.426586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f6972-fabe-41da-9da6-bdc9f538b6c1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:43.427818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48bfad-82e0-48ad-9bef-406fda1c4f50 None admin] RBAC: Authorizing `identity:get_group(group_id=0db9f5507e2a4efe94d8fc33470c310e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:43.429880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48bfad-82e0-48ad-9bef-406fda1c4f50 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:43.431533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 184/370] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:43 2022] GET /identity/v3/groups/0db9f5507e2a4efe94d8fc33470c310e => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:43.432305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48bfad-82e0-48ad-9bef-406fda1c4f50 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:43.457852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:43.476700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76255e6-ef0c-484f-b917-c9c937293d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:43.477697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76255e6-ef0c-484f-b917-c9c937293d29 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:43.478437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76255e6-ef0c-484f-b917-c9c937293d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:43.479045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76255e6-ef0c-484f-b917-c9c937293d29 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:43.479873 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76255e6-ef0c-484f-b917-c9c937293d29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:43.480289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76255e6-ef0c-484f-b917-c9c937293d29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:43.481922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76255e6-ef0c-484f-b917-c9c937293d29 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:43.493245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 187/371] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:43.494175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76255e6-ef0c-484f-b917-c9c937293d29 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:43.558654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48bfad-82e0-48ad-9bef-406fda1c4f50 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:43.607179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0742a-985c-400e-a604-1ba25bb95bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:43.607789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0742a-985c-400e-a604-1ba25bb95bbc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:43.608066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0742a-985c-400e-a604-1ba25bb95bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:43.608343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0742a-985c-400e-a604-1ba25bb95bbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:43.614598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76255e6-ef0c-484f-b917-c9c937293d29 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:43.663454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfbc57-5454-44c3-9402-b47548fae80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:43.664828 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfbc57-5454-44c3-9402-b47548fae80a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:43.665476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfbc57-5454-44c3-9402-b47548fae80a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:43.668882 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:43.669494 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfbc57-5454-44c3-9402-b47548fae80a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:43.670523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfbc57-5454-44c3-9402-b47548fae80a None admin] RBAC: Adding query filter params (group.id=0db9f5507e2a4efe94d8fc33470c310e, role.id=ac5a835695db44918d768db1bbb97589, scope.project.id=cbda193ce0194e0294b307a9296fa907) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:43.671061 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfbc57-5454-44c3-9402-b47548fae80a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:43.673014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfbc57-5454-44c3-9402-b47548fae80a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:43.680655 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 188/372] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Aug 19 16:18:43 2022] GET /identity/v3/role_assignments?group.id=0db9f5507e2a4efe94d8fc33470c310e&scope.project.id=cbda193ce0194e0294b307a9296fa907&role.id=ac5a835695db44918d768db1bbb97589 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:43.684717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfbc57-5454-44c3-9402-b47548fae80a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:43.686576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0742a-985c-400e-a604-1ba25bb95bbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:43.689944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0742a-985c-400e-a604-1ba25bb95bbc None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:43.715289 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:43.807423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfbc57-5454-44c3-9402-b47548fae80a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:43.811212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 185/373] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:43.817360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0742a-985c-400e-a604-1ba25bb95bbc None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:43.846990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201bced-e9d0-4ea4-8c99-223ae757bc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:43.847867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201bced-e9d0-4ea4-8c99-223ae757bc3f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:43.848292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201bced-e9d0-4ea4-8c99-223ae757bc3f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:43.848707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201bced-e9d0-4ea4-8c99-223ae757bc3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:43.917720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:43.926457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201bced-e9d0-4ea4-8c99-223ae757bc3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:43.929642 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201bced-e9d0-4ea4-8c99-223ae757bc3f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:44.021175 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0742a-985c-400e-a604-1ba25bb95bbc None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:44.068193 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:44.087737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 189/374] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:44.096640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a47fa-c3da-4dae-b89d-db301cd07b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:44.097307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a47fa-c3da-4dae-b89d-db301cd07b74 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:44.097587 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a47fa-c3da-4dae-b89d-db301cd07b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:44.097860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a47fa-c3da-4dae-b89d-db301cd07b74 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:44.099004 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a47fa-c3da-4dae-b89d-db301cd07b74 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:44.101034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a47fa-c3da-4dae-b89d-db301cd07b74 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:44.103034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 186/375] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:43 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:44.105324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201bced-e9d0-4ea4-8c99-223ae757bc3f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:44.116653 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a47fa-c3da-4dae-b89d-db301cd07b74 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:44.271618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:44.282549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201bced-e9d0-4ea4-8c99-223ae757bc3f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:44.314982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a47fa-c3da-4dae-b89d-db301cd07b74 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:44.357488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640718b-c2c7-48bd-afef-81306a52eef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:44.358140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640718b-c2c7-48bd-afef-81306a52eef2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:44.358421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640718b-c2c7-48bd-afef-81306a52eef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:44.358589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b686284-ff26-4117-991e-344fc900a371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:44.358714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640718b-c2c7-48bd-afef-81306a52eef2 None admin] PATH_INFO: `/v3/projects/61d450c451d94f27b564ec08cc487161` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:44.359244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b686284-ff26-4117-991e-344fc900a371 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:44.359454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b686284-ff26-4117-991e-344fc900a371 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:44.359724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b686284-ff26-4117-991e-344fc900a371 None admin] PATH_INFO: `/v3/groups/0db9f5507e2a4efe94d8fc33470c310e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:44.359896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640718b-c2c7-48bd-afef-81306a52eef2 None admin] RBAC: Authorizing `identity:get_project(project_id=61d450c451d94f27b564ec08cc487161)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:44.360865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b686284-ff26-4117-991e-344fc900a371 None admin] RBAC: Authorizing `identity:get_group(group_id=0db9f5507e2a4efe94d8fc33470c310e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:44.361321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640718b-c2c7-48bd-afef-81306a52eef2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:44.362288 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b686284-ff26-4117-991e-344fc900a371 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:44.363273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 190/376] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:44 2022] GET /identity/v3/projects/61d450c451d94f27b564ec08cc487161 => generated 344 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:44.364115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 187/377] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:44 2022] GET /identity/v3/groups/0db9f5507e2a4efe94d8fc33470c310e => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:44.368969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640718b-c2c7-48bd-afef-81306a52eef2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:44.373811 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b686284-ff26-4117-991e-344fc900a371 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:44.481572 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:44.535999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640718b-c2c7-48bd-afef-81306a52eef2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:44.577074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321616d-7737-48aa-9985-86588325bbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:44.577768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321616d-7737-48aa-9985-86588325bbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:44.578728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321616d-7737-48aa-9985-86588325bbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:44.579191 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321616d-7737-48aa-9985-86588325bbe5 None admin] PATH_INFO: `/v3/groups/1ccfdae23d3c4ec8b46d7e489c3bc972` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:44.587114 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321616d-7737-48aa-9985-86588325bbe5 None admin] RBAC: Authorizing `identity:get_group(group_id=1ccfdae23d3c4ec8b46d7e489c3bc972)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:44.589057 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321616d-7737-48aa-9985-86588325bbe5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:44.591000 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 191/378] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:44 2022] GET /identity/v3/groups/1ccfdae23d3c4ec8b46d7e489c3bc972 => generated 229 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:44.596848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321616d-7737-48aa-9985-86588325bbe5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:44.602720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b686284-ff26-4117-991e-344fc900a371 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:44.631840 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:44.676107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20ba45-89a5-4924-aa73-64324be6e746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:44.676786 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20ba45-89a5-4924-aa73-64324be6e746 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:44.677068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20ba45-89a5-4924-aa73-64324be6e746 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:44.677336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20ba45-89a5-4924-aa73-64324be6e746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:44.678640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20ba45-89a5-4924-aa73-64324be6e746 None admin] RBAC: Adding query filter params (group.id=0db9f5507e2a4efe94d8fc33470c310e, role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5, scope.project.id=cbda193ce0194e0294b307a9296fa907) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:44.679056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20ba45-89a5-4924-aa73-64324be6e746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:44.680493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20ba45-89a5-4924-aa73-64324be6e746 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:44.696393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 188/379] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Aug 19 16:18:44 2022] GET /identity/v3/role_assignments?group.id=0db9f5507e2a4efe94d8fc33470c310e&scope.project.id=cbda193ce0194e0294b307a9296fa907&role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 260 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:44.735486 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321616d-7737-48aa-9985-86588325bbe5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:44.779795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106b4d4-89a4-4590-b074-d44f2ba475a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:44.780406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106b4d4-89a4-4590-b074-d44f2ba475a7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:44.780682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106b4d4-89a4-4590-b074-d44f2ba475a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:44.780975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106b4d4-89a4-4590-b074-d44f2ba475a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:44.781701 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106b4d4-89a4-4590-b074-d44f2ba475a7 None admin] RBAC: Adding query filter params (group.id=1ccfdae23d3c4ec8b46d7e489c3bc972, role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=61d450c451d94f27b564ec08cc487161) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:44.782005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106b4d4-89a4-4590-b074-d44f2ba475a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:44.783431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106b4d4-89a4-4590-b074-d44f2ba475a7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:44.789499 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 192/380] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Aug 19 16:18:44 2022] GET /identity/v3/role_assignments?group.id=1ccfdae23d3c4ec8b46d7e489c3bc972&scope.project.id=61d450c451d94f27b564ec08cc487161&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:45.097301 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846a47e-05bf-4f41-880b-6a062262a0d0 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:45.097637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846a47e-05bf-4f41-880b-6a062262a0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:45.097832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846a47e-05bf-4f41-880b-6a062262a0d0 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:45.099453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 189/381] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:45.105596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05ea07-055a-4572-864f-7556a306a8aa None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:45.105881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05ea07-055a-4572-864f-7556a306a8aa None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:45.106156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05ea07-055a-4572-864f-7556a306a8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:45.130623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9b05ea07-055a-4572-864f-7556a306a8aa None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:45.132009 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9b05ea07-055a-4572-864f-7556a306a8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:45.219099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 193/382] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:45.611165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8839f03-8563-450c-b7f5-b8882da88983 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:45.611665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8839f03-8563-450c-b7f5-b8882da88983 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:45.611740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8839f03-8563-450c-b7f5-b8882da88983 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:45.613321 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 190/383] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:45.621662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0e133-550d-49c7-b516-d13b9ead66eb None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:45.622061 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0e133-550d-49c7-b516-d13b9ead66eb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:45.622509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0e133-550d-49c7-b516-d13b9ead66eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:45.649816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-47c0e133-550d-49c7-b516-d13b9ead66eb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:45.652808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-47c0e133-550d-49c7-b516-d13b9ead66eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:45.670796 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:45.701123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8839f03-8563-450c-b7f5-b8882da88983 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:45.752355 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 194/384] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:45.890617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8839f03-8563-450c-b7f5-b8882da88983 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:45.917287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:45.939319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd024c-bc85-40a7-b9cf-5dea90eefc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:45.939964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd024c-bc85-40a7-b9cf-5dea90eefc47 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:45.940607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd024c-bc85-40a7-b9cf-5dea90eefc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:45.940607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd024c-bc85-40a7-b9cf-5dea90eefc47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:45.961714 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd024c-bc85-40a7-b9cf-5dea90eefc47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-99fd024c-bc85-40a7-b9cf-5dea90eefc47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:45.967239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:45.968791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:45.969680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:45.969680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:45.969680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:45.969680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:45.969680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:45.969680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:45.969680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:45.969680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:45.969680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:45.969680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:45.969680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:45.969680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:45.970829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 191/385] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:45.974691 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0e133-550d-49c7-b516-d13b9ead66eb None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:46.066097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb7588-e6aa-405b-a9c9-f26474d08f07 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:46.066450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb7588-e6aa-405b-a9c9-f26474d08f07 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:46.066689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb7588-e6aa-405b-a9c9-f26474d08f07 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:46.075127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 192/386] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:46.076775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5413bd2-0c85-497a-aea3-933aaedc625c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:46.077076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5413bd2-0c85-497a-aea3-933aaedc625c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:46.077363 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5413bd2-0c85-497a-aea3-933aaedc625c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:46.117279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b5413bd2-0c85-497a-aea3-933aaedc625c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:46.118495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b5413bd2-0c85-497a-aea3-933aaedc625c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:46.144815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0e133-550d-49c7-b516-d13b9ead66eb None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:46.183749 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e82846-c819-4a29-8a11-75dc63dad7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:46.184373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e82846-c819-4a29-8a11-75dc63dad7ee None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:46.184659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e82846-c819-4a29-8a11-75dc63dad7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:46.184963 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e82846-c819-4a29-8a11-75dc63dad7ee None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:46.185634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e82846-c819-4a29-8a11-75dc63dad7ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:46.186003 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e82846-c819-4a29-8a11-75dc63dad7ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:46.187624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e82846-c819-4a29-8a11-75dc63dad7ee None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:46.199249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 195/387] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:46.200488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e82846-c819-4a29-8a11-75dc63dad7ee None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:46.206317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 193/388] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:46.207266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5413bd2-0c85-497a-aea3-933aaedc625c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:46.219214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:46.315888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e82846-c819-4a29-8a11-75dc63dad7ee None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:46.359312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020a696-bd7a-49d8-96de-f7dd516b36b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:46.359972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020a696-bd7a-49d8-96de-f7dd516b36b6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:46.360258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020a696-bd7a-49d8-96de-f7dd516b36b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:46.360827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020a696-bd7a-49d8-96de-f7dd516b36b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:46.388625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020a696-bd7a-49d8-96de-f7dd516b36b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-1020a696-bd7a-49d8-96de-f7dd516b36b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:46.394020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:46.394937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:46.395721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 196/389] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:46.402805 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020a696-bd7a-49d8-96de-f7dd516b36b6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:46.474917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5413bd2-0c85-497a-aea3-933aaedc625c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:46.493427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:46.514631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:46.524883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020a696-bd7a-49d8-96de-f7dd516b36b6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:46.548888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ff0d0-803a-41d0-af72-fdb557fdc049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:46.549906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ff0d0-803a-41d0-af72-fdb557fdc049 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:46.550411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ff0d0-803a-41d0-af72-fdb557fdc049 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:46.550995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ff0d0-803a-41d0-af72-fdb557fdc049 None admin] PATH_INFO: `/v3/groups/0db9f5507e2a4efe94d8fc33470c310e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:46.553792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ff0d0-803a-41d0-af72-fdb557fdc049 None admin] RBAC: Authorizing `identity:get_group(group_id=0db9f5507e2a4efe94d8fc33470c310e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:46.556938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ff0d0-803a-41d0-af72-fdb557fdc049 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:46.560129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 194/390] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:46 2022] GET /identity/v3/groups/0db9f5507e2a4efe94d8fc33470c310e => generated 226 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:46.561914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ff0d0-803a-41d0-af72-fdb557fdc049 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:46.572873 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08b9f8-5385-4e20-90f2-face413658a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:46.573500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08b9f8-5385-4e20-90f2-face413658a4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:46.573781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08b9f8-5385-4e20-90f2-face413658a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:46.574056 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08b9f8-5385-4e20-90f2-face413658a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:46.574729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08b9f8-5385-4e20-90f2-face413658a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:46.575028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08b9f8-5385-4e20-90f2-face413658a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:46.576458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08b9f8-5385-4e20-90f2-face413658a4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:46.587288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 197/391] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:46.590182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08b9f8-5385-4e20-90f2-face413658a4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:46.720045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08b9f8-5385-4e20-90f2-face413658a4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:46.723059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ff0d0-803a-41d0-af72-fdb557fdc049 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:46.763051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183ab57-a387-4c5d-afba-f5fbee6caaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:46.764066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183ab57-a387-4c5d-afba-f5fbee6caaea None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:46.764547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183ab57-a387-4c5d-afba-f5fbee6caaea None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:46.765303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183ab57-a387-4c5d-afba-f5fbee6caaea None admin] PATH_INFO: `/v3/roles/member` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:46.770094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285128ac-0927-44c6-849a-5fb9d65893aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:46.770731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285128ac-0927-44c6-849a-5fb9d65893aa None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:46.771063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285128ac-0927-44c6-849a-5fb9d65893aa None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:46.771344 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285128ac-0927-44c6-849a-5fb9d65893aa None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:46.772974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285128ac-0927-44c6-849a-5fb9d65893aa None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:46.774603 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285128ac-0927-44c6-849a-5fb9d65893aa None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:46.780908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 198/392] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:46 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:46.782550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285128ac-0927-44c6-849a-5fb9d65893aa None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:46.784916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183ab57-a387-4c5d-afba-f5fbee6caaea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:46.786310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-a183ab57-a387-4c5d-afba-f5fbee6caaea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:46.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:46.800525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:46.801375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:46.801375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:46.801375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:46.801375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:46.801375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:46.801375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:46.801375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:46.801375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:46.801375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:46.801375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:46.801375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:46.801375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:46.806774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 195/393] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:46.808242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183ab57-a387-4c5d-afba-f5fbee6caaea None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:46.874338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:46.919674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285128ac-0927-44c6-849a-5fb9d65893aa None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:46.957917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183ab57-a387-4c5d-afba-f5fbee6caaea None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:46.963567 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944437d9-e786-44d9-9a41-03b3cc59dfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:46.964250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944437d9-e786-44d9-9a41-03b3cc59dfac None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:46.964535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944437d9-e786-44d9-9a41-03b3cc59dfac None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:46.964853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944437d9-e786-44d9-9a41-03b3cc59dfac None admin] PATH_INFO: `/v3/groups/0db9f5507e2a4efe94d8fc33470c310e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:46.965996 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944437d9-e786-44d9-9a41-03b3cc59dfac None admin] RBAC: Authorizing `identity:get_group(group_id=0db9f5507e2a4efe94d8fc33470c310e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:46.967750 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944437d9-e786-44d9-9a41-03b3cc59dfac None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:46.974062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 199/394] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:46 2022] GET /identity/v3/groups/0db9f5507e2a4efe94d8fc33470c310e => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:46.974062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944437d9-e786-44d9-9a41-03b3cc59dfac None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:47.009912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a04b0-f095-45f6-9bea-ed543d5bcef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:47.010869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a04b0-f095-45f6-9bea-ed543d5bcef3 None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:47.011294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a04b0-f095-45f6-9bea-ed543d5bcef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:47.011722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a04b0-f095-45f6-9bea-ed543d5bcef3 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091/groups/0db9f5507e2a4efe94d8fc33470c310e/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:47.014555 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a04b0-f095-45f6-9bea-ed543d5bcef3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9164929bc082468aa5cb93ca3d057091, group_id=0db9f5507e2a4efe94d8fc33470c310e, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:47.016421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a04b0-f095-45f6-9bea-ed543d5bcef3 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:47.040297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 196/395] 10.222.0.55 () {58 vars in 1442 bytes} [Fri Aug 19 16:18:46 2022] PUT /identity/v3/projects/9164929bc082468aa5cb93ca3d057091/groups/0db9f5507e2a4efe94d8fc33470c310e/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:47.040297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a04b0-f095-45f6-9bea-ed543d5bcef3 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:47.108698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:47.191619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:47.195219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944437d9-e786-44d9-9a41-03b3cc59dfac None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:47.195725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a04b0-f095-45f6-9bea-ed543d5bcef3 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:47.248109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c1890-f9d1-4689-a1e9-337ade70cfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:47.248828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c1890-f9d1-4689-a1e9-337ade70cfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:47.249056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c1890-f9d1-4689-a1e9-337ade70cfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:47.249336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c1890-f9d1-4689-a1e9-337ade70cfb3 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:47.251298 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c1890-f9d1-4689-a1e9-337ade70cfb3 None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:47.252794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c1890-f9d1-4689-a1e9-337ade70cfb3 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:47.253326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180cb0f-af76-4e25-8e71-334dd2fed52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:47.253946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180cb0f-af76-4e25-8e71-334dd2fed52d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:47.254234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180cb0f-af76-4e25-8e71-334dd2fed52d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:47.254428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 197/396] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:47 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:47.254590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180cb0f-af76-4e25-8e71-334dd2fed52d None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:47.255248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180cb0f-af76-4e25-8e71-334dd2fed52d None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:47.255556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180cb0f-af76-4e25-8e71-334dd2fed52d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:47.256142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c4014-6c37-443a-bae5-6d7583b08339 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:47.256428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c4014-6c37-443a-bae5-6d7583b08339 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:47.256704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c4014-6c37-443a-bae5-6d7583b08339 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:47.257466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180cb0f-af76-4e25-8e71-334dd2fed52d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:47.261182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 198/397] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:47.263072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c4014-6c37-443a-bae5-6d7583b08339 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:47.266825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 200/398] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:47.268103 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc6ed8-5b64-49d8-bd31-fd93c7236092 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:47.268398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc6ed8-5b64-49d8-bd31-fd93c7236092 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:47.268726 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc6ed8-5b64-49d8-bd31-fd93c7236092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:47.291425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4ebc6ed8-5b64-49d8-bd31-fd93c7236092 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:47.292653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4ebc6ed8-5b64-49d8-bd31-fd93c7236092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:47.372330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 201/399] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:47.373404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc6ed8-5b64-49d8-bd31-fd93c7236092 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:47.395552 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c4014-6c37-443a-bae5-6d7583b08339 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:47.441536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:47.445397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fedec-fde6-4d8f-bd4a-4f2853bd5731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:47.446318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fedec-fde6-4d8f-bd4a-4f2853bd5731 None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:47.446803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fedec-fde6-4d8f-bd4a-4f2853bd5731 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:47.447455 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fedec-fde6-4d8f-bd4a-4f2853bd5731 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091/groups/0db9f5507e2a4efe94d8fc33470c310e/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:47.450662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fedec-fde6-4d8f-bd4a-4f2853bd5731 None admin] RBAC: Authorizing `identity:create_grant(project_id=9164929bc082468aa5cb93ca3d057091, group_id=0db9f5507e2a4efe94d8fc33470c310e, role_id=f7bd4a6c906a4ad3b6d2f003f9c37bc5)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:47.452500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fedec-fde6-4d8f-bd4a-4f2853bd5731 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:47.477630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 199/400] 10.222.0.55 () {58 vars in 1442 bytes} [Fri Aug 19 16:18:47 2022] PUT /identity/v3/projects/9164929bc082468aa5cb93ca3d057091/groups/0db9f5507e2a4efe94d8fc33470c310e/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:47.480136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139d45f-fbc0-46ad-b50e-c1c9bdb2c444 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:47.480609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139d45f-fbc0-46ad-b50e-c1c9bdb2c444 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:47.481343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139d45f-fbc0-46ad-b50e-c1c9bdb2c444 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:47.483920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 200/401] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:47.489499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c27fc-f2ac-4c4d-9e8c-b4dbd4a3d23a None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:47.490037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c27fc-f2ac-4c4d-9e8c-b4dbd4a3d23a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:47.490747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c27fc-f2ac-4c4d-9e8c-b4dbd4a3d23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:47.502861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc6ed8-5b64-49d8-bd31-fd93c7236092 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:47.518572 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-a09c27fc-f2ac-4c4d-9e8c-b4dbd4a3d23a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:47.520296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a09c27fc-f2ac-4c4d-9e8c-b4dbd4a3d23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:47.547592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:47.608135 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 201/402] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:47.609286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00689bdb-39fa-4f9f-9bba-fe0413f6d724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:47.609962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00689bdb-39fa-4f9f-9bba-fe0413f6d724 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:47.610247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00689bdb-39fa-4f9f-9bba-fe0413f6d724 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:47.610526 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00689bdb-39fa-4f9f-9bba-fe0413f6d724 None admin] PATH_INFO: `/v3/groups/0db9f5507e2a4efe94d8fc33470c310e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:47.611802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00689bdb-39fa-4f9f-9bba-fe0413f6d724 None admin] RBAC: Authorizing `identity:get_group(group_id=0db9f5507e2a4efe94d8fc33470c310e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:47.613605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00689bdb-39fa-4f9f-9bba-fe0413f6d724 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:47.616044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 202/403] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:47 2022] GET /identity/v3/groups/0db9f5507e2a4efe94d8fc33470c310e => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:47.621591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c27fc-f2ac-4c4d-9e8c-b4dbd4a3d23a None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:47.656202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00689bdb-39fa-4f9f-9bba-fe0413f6d724 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:47.771501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c27fc-f2ac-4c4d-9e8c-b4dbd4a3d23a None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:47.827151 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00689bdb-39fa-4f9f-9bba-fe0413f6d724 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:47.829168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94cf68-f55d-4571-8596-ffac3ab6c78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:47.829921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94cf68-f55d-4571-8596-ffac3ab6c78e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:47.830239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94cf68-f55d-4571-8596-ffac3ab6c78e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:47.830520 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94cf68-f55d-4571-8596-ffac3ab6c78e None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:47.831694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94cf68-f55d-4571-8596-ffac3ab6c78e None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:47.833164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94cf68-f55d-4571-8596-ffac3ab6c78e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:47.835716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 202/404] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:47 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:47.841173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94cf68-f55d-4571-8596-ffac3ab6c78e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:47.859376 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:47.878356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555b0cb-5be7-4b83-ad6e-5c4332de80a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:47.879005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555b0cb-5be7-4b83-ad6e-5c4332de80a6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:47.879132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:47.879303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555b0cb-5be7-4b83-ad6e-5c4332de80a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:47.879545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555b0cb-5be7-4b83-ad6e-5c4332de80a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:47.901986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555b0cb-5be7-4b83-ad6e-5c4332de80a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4555b0cb-5be7-4b83-ad6e-5c4332de80a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:47.914925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:47.915897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:47.916693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 203/405] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:18:47.921845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555b0cb-5be7-4b83-ad6e-5c4332de80a6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:47.963177 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94cf68-f55d-4571-8596-ffac3ab6c78e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:48.002197 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f563e4b-88e1-4a30-acdd-441926cfa012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:48.002865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f563e4b-88e1-4a30-acdd-441926cfa012 None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:48.003147 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f563e4b-88e1-4a30-acdd-441926cfa012 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:48.003429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f563e4b-88e1-4a30-acdd-441926cfa012 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907/groups/0db9f5507e2a4efe94d8fc33470c310e/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:48.005590 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f563e4b-88e1-4a30-acdd-441926cfa012 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbda193ce0194e0294b307a9296fa907, group_id=0db9f5507e2a4efe94d8fc33470c310e, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:48.008314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f563e4b-88e1-4a30-acdd-441926cfa012 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:48.023127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 203/406] 10.222.0.55 () {58 vars in 1442 bytes} [Fri Aug 19 16:18:47 2022] PUT /identity/v3/projects/cbda193ce0194e0294b307a9296fa907/groups/0db9f5507e2a4efe94d8fc33470c310e/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:48.052182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f563e4b-88e1-4a30-acdd-441926cfa012 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:48.095045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555b0cb-5be7-4b83-ad6e-5c4332de80a6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:48.166742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:48.175777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c95a15-95b7-44fe-93e9-71a1579eb2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:48.176554 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c95a15-95b7-44fe-93e9-71a1579eb2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:48.176738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c95a15-95b7-44fe-93e9-71a1579eb2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:48.177004 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c95a15-95b7-44fe-93e9-71a1579eb2b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:48.177749 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c95a15-95b7-44fe-93e9-71a1579eb2b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:48.178960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c95a15-95b7-44fe-93e9-71a1579eb2b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:48.180393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c95a15-95b7-44fe-93e9-71a1579eb2b9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:48.191392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f563e4b-88e1-4a30-acdd-441926cfa012 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:48.196904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 204/407] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:48.202811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c95a15-95b7-44fe-93e9-71a1579eb2b9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:48.233304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42feeaf8-f243-44a1-9d5e-24ce527e32f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:48.234057 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42feeaf8-f243-44a1-9d5e-24ce527e32f9 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:48.234414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42feeaf8-f243-44a1-9d5e-24ce527e32f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:48.234763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42feeaf8-f243-44a1-9d5e-24ce527e32f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:48.247130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42feeaf8-f243-44a1-9d5e-24ce527e32f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-42feeaf8-f243-44a1-9d5e-24ce527e32f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:48.252377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:48.253300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:48.254062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 204/408] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:18:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:18:48.259876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42feeaf8-f243-44a1-9d5e-24ce527e32f9 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:48.272666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:48.334690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c95a15-95b7-44fe-93e9-71a1579eb2b9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:48.370452 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42feeaf8-f243-44a1-9d5e-24ce527e32f9 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:48.406080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd011ee3-376f-4948-a24f-9d89291926e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:48.406314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf826f-6429-4665-9c1c-d2a7c7919012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:48.406726 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd011ee3-376f-4948-a24f-9d89291926e4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:48.406838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf826f-6429-4665-9c1c-d2a7c7919012 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:48.407054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd011ee3-376f-4948-a24f-9d89291926e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:48.407154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf826f-6429-4665-9c1c-d2a7c7919012 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:48.407312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd011ee3-376f-4948-a24f-9d89291926e4 None admin] PATH_INFO: `/v3/groups/0db9f5507e2a4efe94d8fc33470c310e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:48.407404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf826f-6429-4665-9c1c-d2a7c7919012 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:48.407984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf826f-6429-4665-9c1c-d2a7c7919012 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:48.408283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf826f-6429-4665-9c1c-d2a7c7919012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:48.408381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd011ee3-376f-4948-a24f-9d89291926e4 None admin] RBAC: Authorizing `identity:get_group(group_id=0db9f5507e2a4efe94d8fc33470c310e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:48.409681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf826f-6429-4665-9c1c-d2a7c7919012 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:48.409858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd011ee3-376f-4948-a24f-9d89291926e4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:48.416846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 205/409] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:48 2022] GET /identity/v3/groups/0db9f5507e2a4efe94d8fc33470c310e => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:48.418079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd011ee3-376f-4948-a24f-9d89291926e4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:48.426953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 205/410] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:48.433243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf826f-6429-4665-9c1c-d2a7c7919012 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:48.443381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:48.563413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd011ee3-376f-4948-a24f-9d89291926e4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:48.587536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:48.605383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f69cd-7d4c-4d31-976b-91ed43f5c533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:48.606029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f69cd-7d4c-4d31-976b-91ed43f5c533 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:48.606313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f69cd-7d4c-4d31-976b-91ed43f5c533 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:48.606591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f69cd-7d4c-4d31-976b-91ed43f5c533 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:48.613907 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f69cd-7d4c-4d31-976b-91ed43f5c533 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:48.615408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f69cd-7d4c-4d31-976b-91ed43f5c533 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:48.620119 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 206/411] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:48 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:48.626010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f69cd-7d4c-4d31-976b-91ed43f5c533 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:48.664879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf826f-6429-4665-9c1c-d2a7c7919012 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:48.737777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a76bc-a412-4776-aee5-95538a5f8525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:48.741023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a76bc-a412-4776-aee5-95538a5f8525 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:48.741324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a76bc-a412-4776-aee5-95538a5f8525 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:48.741569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a76bc-a412-4776-aee5-95538a5f8525 None admin] PATH_INFO: `/v3/groups/1ccfdae23d3c4ec8b46d7e489c3bc972` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:48.743728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a76bc-a412-4776-aee5-95538a5f8525 None admin] RBAC: Authorizing `identity:get_group(group_id=1ccfdae23d3c4ec8b46d7e489c3bc972)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:48.745209 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a76bc-a412-4776-aee5-95538a5f8525 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:48.746788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 206/412] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:48 2022] GET /identity/v3/groups/1ccfdae23d3c4ec8b46d7e489c3bc972 => generated 229 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:48.753136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a76bc-a412-4776-aee5-95538a5f8525 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:48.842979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f69cd-7d4c-4d31-976b-91ed43f5c533 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:48.858479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:48.885956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a76bc-a412-4776-aee5-95538a5f8525 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:48.923933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cb772-1f96-4448-92c0-05a3d8ca058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:48.924551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cb772-1f96-4448-92c0-05a3d8ca058b None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:48.924852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cb772-1f96-4448-92c0-05a3d8ca058b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:48.925121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cb772-1f96-4448-92c0-05a3d8ca058b None admin] PATH_INFO: `/v3/projects/61d450c451d94f27b564ec08cc487161` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:48.926239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cb772-1f96-4448-92c0-05a3d8ca058b None admin] RBAC: Authorizing `identity:get_project(project_id=61d450c451d94f27b564ec08cc487161)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:48.927183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d91da2-2976-46f3-b4d9-c5f9867471bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:48.927665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cb772-1f96-4448-92c0-05a3d8ca058b None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:48.928373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d91da2-2976-46f3-b4d9-c5f9867471bc None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:48.930898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 207/413] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:48 2022] GET /identity/v3/projects/61d450c451d94f27b564ec08cc487161 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:48.931228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d91da2-2976-46f3-b4d9-c5f9867471bc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:48.931540 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d91da2-2976-46f3-b4d9-c5f9867471bc None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907/groups/0db9f5507e2a4efe94d8fc33470c310e/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:48.934565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d91da2-2976-46f3-b4d9-c5f9867471bc None admin] RBAC: Authorizing `identity:create_grant(project_id=cbda193ce0194e0294b307a9296fa907, group_id=0db9f5507e2a4efe94d8fc33470c310e, role_id=f7bd4a6c906a4ad3b6d2f003f9c37bc5)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:48.935568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cb772-1f96-4448-92c0-05a3d8ca058b None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:48.939666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d91da2-2976-46f3-b4d9-c5f9867471bc None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:48.977461 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 207/414] 10.222.0.55 () {58 vars in 1442 bytes} [Fri Aug 19 16:18:48 2022] PUT /identity/v3/projects/cbda193ce0194e0294b307a9296fa907/groups/0db9f5507e2a4efe94d8fc33470c310e/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:49.056190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cb772-1f96-4448-92c0-05a3d8ca058b None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:49.095287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9f40d-d721-43f9-ab8c-f344d29e9615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:49.096376 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9f40d-d721-43f9-ab8c-f344d29e9615 None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:49.096887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9f40d-d721-43f9-ab8c-f344d29e9615 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:49.097357 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9f40d-d721-43f9-ab8c-f344d29e9615 None admin] PATH_INFO: `/v3/projects/61d450c451d94f27b564ec08cc487161/groups/1ccfdae23d3c4ec8b46d7e489c3bc972/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:49.099786 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9f40d-d721-43f9-ab8c-f344d29e9615 None admin] RBAC: Authorizing `identity:create_grant(project_id=61d450c451d94f27b564ec08cc487161, group_id=1ccfdae23d3c4ec8b46d7e489c3bc972, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:49.101695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9f40d-d721-43f9-ab8c-f344d29e9615 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:49.124493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 208/415] 10.222.0.55 () {58 vars in 1442 bytes} [Fri Aug 19 16:18:48 2022] PUT /identity/v3/projects/61d450c451d94f27b564ec08cc487161/groups/1ccfdae23d3c4ec8b46d7e489c3bc972/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:18:49.315668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66016c-3ea1-42e4-b05a-4527635503df None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:49.316373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66016c-3ea1-42e4-b05a-4527635503df None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:49.316909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66016c-3ea1-42e4-b05a-4527635503df None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:49.318567 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 208/416] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:49.325387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526aae5-ba91-42d4-a280-9cf4f672b588 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:49.325943 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526aae5-ba91-42d4-a280-9cf4f672b588 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:49.326421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526aae5-ba91-42d4-a280-9cf4f672b588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:49.332970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:49.347671 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-7526aae5-ba91-42d4-a280-9cf4f672b588 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:49.349186 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7526aae5-ba91-42d4-a280-9cf4f672b588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:49.420974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 209/417] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:49.674779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee782419-dfd6-468e-97b0-0c35bfe45bc1 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:49.675396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee782419-dfd6-468e-97b0-0c35bfe45bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:49.675866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee782419-dfd6-468e-97b0-0c35bfe45bc1 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:49.677687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 209/418] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:49.683246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6c4ef-5e4e-43af-b1c6-169448d05abb None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:49.683817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6c4ef-5e4e-43af-b1c6-169448d05abb None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:49.684296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6c4ef-5e4e-43af-b1c6-169448d05abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:49.713748 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-8bf6c4ef-5e4e-43af-b1c6-169448d05abb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:49.715304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8bf6c4ef-5e4e-43af-b1c6-169448d05abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:49.789104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 210/419] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:49.800426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee782419-dfd6-468e-97b0-0c35bfe45bc1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:49.865738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:49.938108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee782419-dfd6-468e-97b0-0c35bfe45bc1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:49.982877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d3b3c-4d23-42fa-8658-63f358f618ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:49.983528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d3b3c-4d23-42fa-8658-63f358f618ea None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:49.983809 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d3b3c-4d23-42fa-8658-63f358f618ea None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:49.984081 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d3b3c-4d23-42fa-8658-63f358f618ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:49.999356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d3b3c-4d23-42fa-8658-63f358f618ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-536d3b3c-4d23-42fa-8658-63f358f618ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:50.004131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:50.005016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:50.005804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:50.005804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:50.005804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:50.005804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:50.005804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:50.005804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:50.005804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:50.005804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:50.005804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:50.005804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:50.005804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:50.005804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:50.007400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 210/420] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:50.013861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6c4ef-5e4e-43af-b1c6-169448d05abb None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:50.071078 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d3b3c-4d23-42fa-8658-63f358f618ea None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:50.105149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:50.175568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6c4ef-5e4e-43af-b1c6-169448d05abb None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:50.214281 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61db50b-5a95-416a-abf8-312bc69e7c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:50.214945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61db50b-5a95-416a-abf8-312bc69e7c97 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:50.215230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61db50b-5a95-416a-abf8-312bc69e7c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:50.215501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61db50b-5a95-416a-abf8-312bc69e7c97 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:50.216162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61db50b-5a95-416a-abf8-312bc69e7c97 None admin] RBAC: Adding query filter params (name=member) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:50.216464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61db50b-5a95-416a-abf8-312bc69e7c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:50.218107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61db50b-5a95-416a-abf8-312bc69e7c97 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:50.232841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 211/421] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:50.238647 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61db50b-5a95-416a-abf8-312bc69e7c97 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:50.255683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d3b3c-4d23-42fa-8658-63f358f618ea None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:50.274261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:50.323302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cad1fa-4e37-4813-bc9e-3dc3a2a00ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:50.324199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cad1fa-4e37-4813-bc9e-3dc3a2a00ceb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:50.325768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cad1fa-4e37-4813-bc9e-3dc3a2a00ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:50.326062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cad1fa-4e37-4813-bc9e-3dc3a2a00ceb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:50.346013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cad1fa-4e37-4813-bc9e-3dc3a2a00ceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-05cad1fa-4e37-4813-bc9e-3dc3a2a00ceb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:50.352697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:50.353854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:50.354627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:50.354627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:50.354627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:50.354627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:50.354627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:50.354627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:50.354627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:50.354627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:50.354627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:50.354627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:50.354627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:50.354627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:50.355665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 211/422] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:50.363188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cad1fa-4e37-4813-bc9e-3dc3a2a00ceb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:50.431609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61db50b-5a95-416a-abf8-312bc69e7c97 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:50.497174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:50.513039 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4778fb-0938-4b0a-80a6-3fed52793387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:50.513659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4778fb-0938-4b0a-80a6-3fed52793387 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:50.513935 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4778fb-0938-4b0a-80a6-3fed52793387 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:50.514204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4778fb-0938-4b0a-80a6-3fed52793387 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:50.564150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cad1fa-4e37-4813-bc9e-3dc3a2a00ceb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:50.601467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4778fb-0938-4b0a-80a6-3fed52793387 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:50.603642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4778fb-0938-4b0a-80a6-3fed52793387 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:50.606148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5113db-4806-4429-961c-268f876bc695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:50.607523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5113db-4806-4429-961c-268f876bc695 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:50.607805 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5113db-4806-4429-961c-268f876bc695 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:50.608082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5113db-4806-4429-961c-268f876bc695 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:50.608745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5113db-4806-4429-961c-268f876bc695 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:50.609068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5113db-4806-4429-961c-268f876bc695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:50.610435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5113db-4806-4429-961c-268f876bc695 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:50.624909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 212/423] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:50.626412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2398e1-4784-4101-9b7a-3e63b8c6bf2b None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:50.626694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2398e1-4784-4101-9b7a-3e63b8c6bf2b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:50.626981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2398e1-4784-4101-9b7a-3e63b8c6bf2b None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:50.628319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 213/424] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:50.630306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2398e1-4784-4101-9b7a-3e63b8c6bf2b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:50.674458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:50.744842 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 212/425] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:50.746509 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7df9b1-aa5f-4c3c-bd50-2606521e0d54 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:50.747011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7df9b1-aa5f-4c3c-bd50-2606521e0d54 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:50.747543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7df9b1-aa5f-4c3c-bd50-2606521e0d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:50.759728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2398e1-4784-4101-9b7a-3e63b8c6bf2b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:50.767187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ca7df9b1-aa5f-4c3c-bd50-2606521e0d54 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:50.768715 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ca7df9b1-aa5f-4c3c-bd50-2606521e0d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:50.789792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:50.800597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a2ffa-966f-4da4-9db0-b401210dffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:50.801396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a2ffa-966f-4da4-9db0-b401210dffbb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:50.801678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a2ffa-966f-4da4-9db0-b401210dffbb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:50.801949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a2ffa-966f-4da4-9db0-b401210dffbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:50.856789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 213/426] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:50.856789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7df9b1-aa5f-4c3c-bd50-2606521e0d54 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:50.911797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a2ffa-966f-4da4-9db0-b401210dffbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:50.916156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a2ffa-966f-4da4-9db0-b401210dffbb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:50.976863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:51.025675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7df9b1-aa5f-4c3c-bd50-2606521e0d54 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:51.090008 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09433aa-16db-4c23-ba32-f4564dd611e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:51.092062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09433aa-16db-4c23-ba32-f4564dd611e4 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.092723 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09433aa-16db-4c23-ba32-f4564dd611e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.093893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09433aa-16db-4c23-ba32-f4564dd611e4 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.097991 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09433aa-16db-4c23-ba32-f4564dd611e4 None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:51.100200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09433aa-16db-4c23-ba32-f4564dd611e4 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:51.102306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 214/427] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:50 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:51.109194 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09433aa-16db-4c23-ba32-f4564dd611e4 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:51.126811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 214/428] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:51.135904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a2ffa-966f-4da4-9db0-b401210dffbb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:51.162049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:51.252693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09433aa-16db-4c23-ba32-f4564dd611e4 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:51.273224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a2ffa-966f-4da4-9db0-b401210dffbb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:51.292684 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ecb9-3eb3-4a3a-88b8-bcc64c996748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:51.293948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ecb9-3eb3-4a3a-88b8-bcc64c996748 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.294831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ecb9-3eb3-4a3a-88b8-bcc64c996748 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.295312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ecb9-3eb3-4a3a-88b8-bcc64c996748 None admin] PATH_INFO: `/v3/groups/0db9f5507e2a4efe94d8fc33470c310e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.296637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428ecb9-3eb3-4a3a-88b8-bcc64c996748 None admin] RBAC: Authorizing `identity:get_group(group_id=0db9f5507e2a4efe94d8fc33470c310e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:51.299125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428ecb9-3eb3-4a3a-88b8-bcc64c996748 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:51.302004 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 215/429] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:51 2022] GET /identity/v3/groups/0db9f5507e2a4efe94d8fc33470c310e => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:51.303624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ecb9-3eb3-4a3a-88b8-bcc64c996748 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:51.310102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:51.317693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08440f73-0e6a-4238-900b-79e0f0d4df08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:51.319360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08440f73-0e6a-4238-900b-79e0f0d4df08 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.319620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08440f73-0e6a-4238-900b-79e0f0d4df08 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.319863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08440f73-0e6a-4238-900b-79e0f0d4df08 None admin] PATH_INFO: `/v3/projects/9164929bc082468aa5cb93ca3d057091` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.321481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08440f73-0e6a-4238-900b-79e0f0d4df08 None admin] RBAC: Authorizing `identity:get_project(project_id=9164929bc082468aa5cb93ca3d057091)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:51.322941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08440f73-0e6a-4238-900b-79e0f0d4df08 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:51.330030 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 215/430] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:51 2022] GET /identity/v3/projects/9164929bc082468aa5cb93ca3d057091 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:51.331010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08440f73-0e6a-4238-900b-79e0f0d4df08 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:51.414729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:51.425089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ecb9-3eb3-4a3a-88b8-bcc64c996748 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:51.434658 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08440f73-0e6a-4238-900b-79e0f0d4df08 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:51.458480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39468e3e-6082-4eec-921e-a18bdcf82129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:51.459623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468e3e-6082-4eec-921e-a18bdcf82129 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.459900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468e3e-6082-4eec-921e-a18bdcf82129 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.460172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468e3e-6082-4eec-921e-a18bdcf82129 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.468766 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813fea1-44be-4721-98c4-4db1ec5d022f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:51.469396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813fea1-44be-4721-98c4-4db1ec5d022f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.469685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813fea1-44be-4721-98c4-4db1ec5d022f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.469953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813fea1-44be-4721-98c4-4db1ec5d022f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.470257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39468e3e-6082-4eec-921e-a18bdcf82129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:51.470677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813fea1-44be-4721-98c4-4db1ec5d022f None admin] RBAC: Adding query filter params (group.id=0db9f5507e2a4efe94d8fc33470c310e, role.id=ac5a835695db44918d768db1bbb97589, scope.project.id=9164929bc082468aa5cb93ca3d057091) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:51.470983 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813fea1-44be-4721-98c4-4db1ec5d022f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:51.472412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813fea1-44be-4721-98c4-4db1ec5d022f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-39468e3e-6082-4eec-921e-a18bdcf82129 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:51.476015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:51.477046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:51.478396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:51.478396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:51.478396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:51.478396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:51.478396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:51.478396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:51.478396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:51.478396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:51.478396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:51.478396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:51.478396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 16:18:51.478396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:51.479229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 216/431] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Aug 19 16:18:51 2022] GET /identity/v3/role_assignments?group.id=0db9f5507e2a4efe94d8fc33470c310e&scope.project.id=9164929bc082468aa5cb93ca3d057091&role.id=ac5a835695db44918d768db1bbb97589 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:51.479862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813fea1-44be-4721-98c4-4db1ec5d022f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:51.481120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 216/432] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:51.482729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056eb2a0-51a2-4889-ab3f-2a520fc693a8 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.483025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056eb2a0-51a2-4889-ab3f-2a520fc693a8 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.483300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056eb2a0-51a2-4889-ab3f-2a520fc693a8 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.484648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 217/433] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:51.485768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec338dc-d3b5-49b7-9b67-69c8cb92e32e None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.486125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec338dc-d3b5-49b7-9b67-69c8cb92e32e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.486407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec338dc-d3b5-49b7-9b67-69c8cb92e32e None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.492489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 218/434] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:51.492489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec338dc-d3b5-49b7-9b67-69c8cb92e32e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:51.562419 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:51.606729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813fea1-44be-4721-98c4-4db1ec5d022f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:51.607742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec338dc-d3b5-49b7-9b67-69c8cb92e32e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:51.650778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3b187-f0bf-493b-ac5a-a64c370baa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:51.651290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3b187-f0bf-493b-ac5a-a64c370baa69 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.652800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3b187-f0bf-493b-ac5a-a64c370baa69 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.652800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3b187-f0bf-493b-ac5a-a64c370baa69 None admin] PATH_INFO: `/v3/groups/0db9f5507e2a4efe94d8fc33470c310e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.652999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3b187-f0bf-493b-ac5a-a64c370baa69 None admin] RBAC: Authorizing `identity:get_group(group_id=0db9f5507e2a4efe94d8fc33470c310e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:51.654450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3b187-f0bf-493b-ac5a-a64c370baa69 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:51.655095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6535ce-ee7d-4864-b424-19cabf3f3f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:51.655966 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6535ce-ee7d-4864-b424-19cabf3f3f48 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.656594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6535ce-ee7d-4864-b424-19cabf3f3f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.656685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 217/435] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:51 2022] GET /identity/v3/groups/0db9f5507e2a4efe94d8fc33470c310e => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:51.656785 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6535ce-ee7d-4864-b424-19cabf3f3f48 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.657154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6535ce-ee7d-4864-b424-19cabf3f3f48 None admin] RBAC: Adding query filter params (name=member) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:51.658104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6535ce-ee7d-4864-b424-19cabf3f3f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:51.658924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6535ce-ee7d-4864-b424-19cabf3f3f48 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:51.664320 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6219c-e6e0-4dec-b8cd-876d66cde20d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.664320 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6219c-e6e0-4dec-b8cd-876d66cde20d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.664320 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6219c-e6e0-4dec-b8cd-876d66cde20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.674425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 219/436] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:18:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:51.674877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d7df7-9e57-484a-8f86-6c241d0025b2 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.675162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d7df7-9e57-484a-8f86-6c241d0025b2 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.675435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d7df7-9e57-484a-8f86-6c241d0025b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.683960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5cc6219c-e6e0-4dec-b8cd-876d66cde20d None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:51.685205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5cc6219c-e6e0-4dec-b8cd-876d66cde20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:51.697344 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-fe1d7df7-9e57-484a-8f86-6c241d0025b2 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:51.698551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-fe1d7df7-9e57-484a-8f86-6c241d0025b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:51.707659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:51.767751 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 218/437] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:51.768608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6219c-e6e0-4dec-b8cd-876d66cde20d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:51.789073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 220/438] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:51.790279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d7df7-9e57-484a-8f86-6c241d0025b2 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:51.854069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:51.891559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6219c-e6e0-4dec-b8cd-876d66cde20d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:51.896326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d7df7-9e57-484a-8f86-6c241d0025b2 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:51.930045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d70d41-6c16-4733-9be9-df5accc98855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:51.930706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d70d41-6c16-4733-9be9-df5accc98855 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.930935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d70d41-6c16-4733-9be9-df5accc98855 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.931208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d70d41-6c16-4733-9be9-df5accc98855 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.931949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d70d41-6c16-4733-9be9-df5accc98855 None admin] RBAC: Adding query filter params (group.id=0db9f5507e2a4efe94d8fc33470c310e, role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5, scope.project.id=9164929bc082468aa5cb93ca3d057091) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:51.932247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d70d41-6c16-4733-9be9-df5accc98855 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:51.933745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d70d41-6c16-4733-9be9-df5accc98855 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:51.934557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c12e63-cc5a-4920-ba1b-ee787ff2f6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:51.935368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c12e63-cc5a-4920-ba1b-ee787ff2f6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:51.935831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c12e63-cc5a-4920-ba1b-ee787ff2f6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:51.936300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c12e63-cc5a-4920-ba1b-ee787ff2f6d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:51.941406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 219/439] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Aug 19 16:18:51 2022] GET /identity/v3/role_assignments?group.id=0db9f5507e2a4efe94d8fc33470c310e&scope.project.id=9164929bc082468aa5cb93ca3d057091&role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:51.962540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:52.054570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c12e63-cc5a-4920-ba1b-ee787ff2f6d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:52.056043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c12e63-cc5a-4920-ba1b-ee787ff2f6d3 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:52.063252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d70d41-6c16-4733-9be9-df5accc98855 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:52.176111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 221/440] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:52.176916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c12e63-cc5a-4920-ba1b-ee787ff2f6d3 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:52.188409 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d70d41-6c16-4733-9be9-df5accc98855 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:52.218144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:52.227261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:52.230834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57028fd-7cf8-42f9-a43f-a16bbcadebdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:52.231472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57028fd-7cf8-42f9-a43f-a16bbcadebdd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:52.231735 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57028fd-7cf8-42f9-a43f-a16bbcadebdd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:52.232004 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57028fd-7cf8-42f9-a43f-a16bbcadebdd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:52.243562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57028fd-7cf8-42f9-a43f-a16bbcadebdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-e57028fd-7cf8-42f9-a43f-a16bbcadebdd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:52.248702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:52.249575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:52.250343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:52.250343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:52.250343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:52.250343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:52.250343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:52.250343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:52.250343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:52.250343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:52.250343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:52.250343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:52.250343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:52.250343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:52.251629 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 220/441] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:18:52.251892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57028fd-7cf8-42f9-a43f-a16bbcadebdd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:52.297697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c12e63-cc5a-4920-ba1b-ee787ff2f6d3 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:52.340262 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6c027-7573-4607-83d1-35ea3963bcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:52.340921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6c027-7573-4607-83d1-35ea3963bcfc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:52.341199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6c027-7573-4607-83d1-35ea3963bcfc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:52.341468 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6c027-7573-4607-83d1-35ea3963bcfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:52.351872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6c027-7573-4607-83d1-35ea3963bcfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-72a6c027-7573-4607-83d1-35ea3963bcfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:52.357041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:52.357922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:52.358711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 222/442] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:18:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:52.359351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6c027-7573-4607-83d1-35ea3963bcfc None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:52.370827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57028fd-7cf8-42f9-a43f-a16bbcadebdd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:52.404653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f7e50-855d-4bec-b786-a4d09b442b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:52.405305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f7e50-855d-4bec-b786-a4d09b442b17 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:52.405597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f7e50-855d-4bec-b786-a4d09b442b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:52.405853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f7e50-855d-4bec-b786-a4d09b442b17 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:52.406983 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f7e50-855d-4bec-b786-a4d09b442b17 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:52.408434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f7e50-855d-4bec-b786-a4d09b442b17 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:52.410071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 221/443] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:52 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:52.410822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f7e50-855d-4bec-b786-a4d09b442b17 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:52.460705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6c027-7573-4607-83d1-35ea3963bcfc None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:52.462164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:52.463690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:52.498217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc554c54-beaf-4587-897f-c00d10f3cb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:52.498819 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc554c54-beaf-4587-897f-c00d10f3cb99 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:52.499097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc554c54-beaf-4587-897f-c00d10f3cb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:52.499369 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc554c54-beaf-4587-897f-c00d10f3cb99 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:52.500021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc554c54-beaf-4587-897f-c00d10f3cb99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:52.500318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc554c54-beaf-4587-897f-c00d10f3cb99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:52.501726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc554c54-beaf-4587-897f-c00d10f3cb99 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:52.509684 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 223/444] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:52.510463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc554c54-beaf-4587-897f-c00d10f3cb99 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:52.511851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f7e50-855d-4bec-b786-a4d09b442b17 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:52.547460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179a99f-b34e-42e1-ab55-8f31640f575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:52.548069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179a99f-b34e-42e1-ab55-8f31640f575a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:52.548351 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179a99f-b34e-42e1-ab55-8f31640f575a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:52.548625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179a99f-b34e-42e1-ab55-8f31640f575a None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:52.549317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179a99f-b34e-42e1-ab55-8f31640f575a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:52.549622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179a99f-b34e-42e1-ab55-8f31640f575a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:52.550991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179a99f-b34e-42e1-ab55-8f31640f575a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:52.559048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 222/445] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:52.559654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179a99f-b34e-42e1-ab55-8f31640f575a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:52.614529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc554c54-beaf-4587-897f-c00d10f3cb99 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:52.650186 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145d601-f288-48c8-94d0-9732ad730280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:52.650845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145d601-f288-48c8-94d0-9732ad730280 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:52.651129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145d601-f288-48c8-94d0-9732ad730280 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:52.651409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145d601-f288-48c8-94d0-9732ad730280 None admin] PATH_INFO: `/v3/groups/0db9f5507e2a4efe94d8fc33470c310e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:52.652546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145d601-f288-48c8-94d0-9732ad730280 None admin] RBAC: Authorizing `identity:get_group(group_id=0db9f5507e2a4efe94d8fc33470c310e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:52.654041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145d601-f288-48c8-94d0-9732ad730280 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:52.660304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 224/446] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:52 2022] GET /identity/v3/groups/0db9f5507e2a4efe94d8fc33470c310e => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:52.660304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145d601-f288-48c8-94d0-9732ad730280 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:52.664410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179a99f-b34e-42e1-ab55-8f31640f575a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:52.678840 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:52.687142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:52.700207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a3239-f040-4f71-b5e4-ea709cced5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:52.700846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a3239-f040-4f71-b5e4-ea709cced5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:52.701126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a3239-f040-4f71-b5e4-ea709cced5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:52.701404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a3239-f040-4f71-b5e4-ea709cced5d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:52.757898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145d601-f288-48c8-94d0-9732ad730280 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:52.769372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a3239-f040-4f71-b5e4-ea709cced5d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:52.770830 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a3239-f040-4f71-b5e4-ea709cced5d7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:52.791740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957ab70-c020-4e9c-bdf1-10edbb303ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:52.792332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957ab70-c020-4e9c-bdf1-10edbb303ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:52.792607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957ab70-c020-4e9c-bdf1-10edbb303ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:52.792906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957ab70-c020-4e9c-bdf1-10edbb303ba9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:52.858394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957ab70-c020-4e9c-bdf1-10edbb303ba9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:52.859839 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957ab70-c020-4e9c-bdf1-10edbb303ba9 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:52.867767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 223/447] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:52.868603 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a3239-f040-4f71-b5e4-ea709cced5d7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:52.883426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:52.893480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:52.962567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 225/448] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:18:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:18:52.963408 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957ab70-c020-4e9c-bdf1-10edbb303ba9 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:52.965894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a3239-f040-4f71-b5e4-ea709cced5d7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:52.999045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d7136-e173-4c53-ae5d-ea795550b730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:52.999637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d7136-e173-4c53-ae5d-ea795550b730 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:52.999909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d7136-e173-4c53-ae5d-ea795550b730 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:53.000178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d7136-e173-4c53-ae5d-ea795550b730 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:53.000921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d7136-e173-4c53-ae5d-ea795550b730 None admin] RBAC: Adding query filter params (group.id=0db9f5507e2a4efe94d8fc33470c310e, role.id=ac5a835695db44918d768db1bbb97589, scope.project.id=cbda193ce0194e0294b307a9296fa907) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:53.001223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d7136-e173-4c53-ae5d-ea795550b730 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:53.002639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d7136-e173-4c53-ae5d-ea795550b730 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:53.008952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 224/449] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Aug 19 16:18:52 2022] GET /identity/v3/role_assignments?group.id=0db9f5507e2a4efe94d8fc33470c310e&scope.project.id=cbda193ce0194e0294b307a9296fa907&role.id=ac5a835695db44918d768db1bbb97589 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:53.009670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d7136-e173-4c53-ae5d-ea795550b730 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:53.069932 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957ab70-c020-4e9c-bdf1-10edbb303ba9 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:53.101797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:53.106428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:53.108920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398efe9-0737-4e08-b76c-2afe4759bed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:53.109597 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398efe9-0737-4e08-b76c-2afe4759bed5 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:53.109883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398efe9-0737-4e08-b76c-2afe4759bed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:53.110155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398efe9-0737-4e08-b76c-2afe4759bed5 None admin] PATH_INFO: `/v3/projects/cbda193ce0194e0294b307a9296fa907` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:53.111323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398efe9-0737-4e08-b76c-2afe4759bed5 None admin] RBAC: Authorizing `identity:get_project(project_id=cbda193ce0194e0294b307a9296fa907)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:53.112843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398efe9-0737-4e08-b76c-2afe4759bed5 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:53.114729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 226/450] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:52 2022] GET /identity/v3/projects/cbda193ce0194e0294b307a9296fa907 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:53.120710 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398efe9-0737-4e08-b76c-2afe4759bed5 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:53.134177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d7136-e173-4c53-ae5d-ea795550b730 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:53.175656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70218fdb-9b07-4135-b887-54f6de1fdeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:53.176295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70218fdb-9b07-4135-b887-54f6de1fdeb5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:53.176574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70218fdb-9b07-4135-b887-54f6de1fdeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:53.177010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70218fdb-9b07-4135-b887-54f6de1fdeb5 None admin] PATH_INFO: `/v3/projects/61d450c451d94f27b564ec08cc487161` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:53.178782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70218fdb-9b07-4135-b887-54f6de1fdeb5 None admin] RBAC: Authorizing `identity:get_project(project_id=61d450c451d94f27b564ec08cc487161)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:53.180219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70218fdb-9b07-4135-b887-54f6de1fdeb5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:53.182042 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 225/451] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:18:53 2022] GET /identity/v3/projects/61d450c451d94f27b564ec08cc487161 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:53.188240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70218fdb-9b07-4135-b887-54f6de1fdeb5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:53.243379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398efe9-0737-4e08-b76c-2afe4759bed5 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:53.284893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6819d93-8ce0-45a5-a89a-e853dc9c6568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:53.285469 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6819d93-8ce0-45a5-a89a-e853dc9c6568 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:53.285745 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6819d93-8ce0-45a5-a89a-e853dc9c6568 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:53.286015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6819d93-8ce0-45a5-a89a-e853dc9c6568 None admin] PATH_INFO: `/v3/groups/0db9f5507e2a4efe94d8fc33470c310e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:53.287852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6819d93-8ce0-45a5-a89a-e853dc9c6568 None admin] RBAC: Authorizing `identity:get_group(group_id=0db9f5507e2a4efe94d8fc33470c310e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:53.289285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6819d93-8ce0-45a5-a89a-e853dc9c6568 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:53.290854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 227/452] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:53 2022] GET /identity/v3/groups/0db9f5507e2a4efe94d8fc33470c310e => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:53.296674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6819d93-8ce0-45a5-a89a-e853dc9c6568 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:53.314173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70218fdb-9b07-4135-b887-54f6de1fdeb5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:53.354860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dd841-3988-4d4f-a8bc-263666c95181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:53.355858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dd841-3988-4d4f-a8bc-263666c95181 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:53.356367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dd841-3988-4d4f-a8bc-263666c95181 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:53.356886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dd841-3988-4d4f-a8bc-263666c95181 None admin] PATH_INFO: `/v3/groups/1ccfdae23d3c4ec8b46d7e489c3bc972` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:53.358039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77dd841-3988-4d4f-a8bc-263666c95181 None admin] RBAC: Authorizing `identity:get_group(group_id=1ccfdae23d3c4ec8b46d7e489c3bc972)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:53.359501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77dd841-3988-4d4f-a8bc-263666c95181 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:53.361415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 226/453] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Aug 19 16:18:53 2022] GET /identity/v3/groups/1ccfdae23d3c4ec8b46d7e489c3bc972 => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:53.368099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dd841-3988-4d4f-a8bc-263666c95181 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:53.379034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:53.385619 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:53.412975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6819d93-8ce0-45a5-a89a-e853dc9c6568 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:53.447070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2b8e5-8513-49b6-a31d-e05023f739e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:53.447677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2b8e5-8513-49b6-a31d-e05023f739e8 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:53.447964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2b8e5-8513-49b6-a31d-e05023f739e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:53.448238 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2b8e5-8513-49b6-a31d-e05023f739e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:53.448991 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2b8e5-8513-49b6-a31d-e05023f739e8 None admin] RBAC: Adding query filter params (group.id=0db9f5507e2a4efe94d8fc33470c310e, role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5, scope.project.id=cbda193ce0194e0294b307a9296fa907) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:53.449298 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2b8e5-8513-49b6-a31d-e05023f739e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:53.450730 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2b8e5-8513-49b6-a31d-e05023f739e8 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:53.456393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 228/454] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Aug 19 16:18:53 2022] GET /identity/v3/role_assignments?group.id=0db9f5507e2a4efe94d8fc33470c310e&scope.project.id=cbda193ce0194e0294b307a9296fa907&role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:53.469861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dd841-3988-4d4f-a8bc-263666c95181 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:53.506490 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca605c-b13d-45f4-9e44-2b9a4e01de64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:53.507117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca605c-b13d-45f4-9e44-2b9a4e01de64 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:53.507398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca605c-b13d-45f4-9e44-2b9a4e01de64 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:53.507671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca605c-b13d-45f4-9e44-2b9a4e01de64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:53.508416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca605c-b13d-45f4-9e44-2b9a4e01de64 None admin] RBAC: Adding query filter params (group.id=1ccfdae23d3c4ec8b46d7e489c3bc972, role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=61d450c451d94f27b564ec08cc487161) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:53.508724 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca605c-b13d-45f4-9e44-2b9a4e01de64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:53.510250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca605c-b13d-45f4-9e44-2b9a4e01de64 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:53.516782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 227/455] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Aug 19 16:18:53 2022] GET /identity/v3/role_assignments?group.id=1ccfdae23d3c4ec8b46d7e489c3bc972&scope.project.id=61d450c451d94f27b564ec08cc487161&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:55.861261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761df5-7942-4209-96f9-28079aedb9ea None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:55.862145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761df5-7942-4209-96f9-28079aedb9ea None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:55.862577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761df5-7942-4209-96f9-28079aedb9ea None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:55.864355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 229/456] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:55.870096 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96157f4b-1f50-4024-a02f-4c355ed0370a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:55.870503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96157f4b-1f50-4024-a02f-4c355ed0370a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:55.870703 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96157f4b-1f50-4024-a02f-4c355ed0370a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:55.894116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-96157f4b-1f50-4024-a02f-4c355ed0370a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:55.895631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-96157f4b-1f50-4024-a02f-4c355ed0370a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:55.995363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 228/457] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:56.151981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b1cb7-5e58-46b9-ad26-675973de2afe None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:56.152303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b1cb7-5e58-46b9-ad26-675973de2afe None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:56.152539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b1cb7-5e58-46b9-ad26-675973de2afe None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:56.154124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 230/458] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:56.159370 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7e3e2-5b41-4a75-a97d-442b4f8e7429 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:56.159886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7e3e2-5b41-4a75-a97d-442b4f8e7429 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:56.160359 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7e3e2-5b41-4a75-a97d-442b4f8e7429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:56.189720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-eda7e3e2-5b41-4a75-a97d-442b4f8e7429 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:56.191267 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-eda7e3e2-5b41-4a75-a97d-442b4f8e7429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:56.193691 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:56.285676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 229/459] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:56.364816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b1cb7-5e58-46b9-ad26-675973de2afe None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:56.509578 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b1cb7-5e58-46b9-ad26-675973de2afe None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:56.556278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:56.563197 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5732e71-db0e-4bb9-be80-c89cea1ace52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:56.563850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5732e71-db0e-4bb9-be80-c89cea1ace52 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:56.564133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5732e71-db0e-4bb9-be80-c89cea1ace52 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:56.564409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5732e71-db0e-4bb9-be80-c89cea1ace52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:56.579646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5732e71-db0e-4bb9-be80-c89cea1ace52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:56.581140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5732e71-db0e-4bb9-be80-c89cea1ace52 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:56.584693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e170efa-0354-4f24-923f-6d391c13eb14 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:56.585247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e170efa-0354-4f24-923f-6d391c13eb14 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:56.585717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e170efa-0354-4f24-923f-6d391c13eb14 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:56.591217 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 230/460] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:56.592592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937bfd1-6000-42f3-b503-9272a068a7d6 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:56.593106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937bfd1-6000-42f3-b503-9272a068a7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:56.593597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937bfd1-6000-42f3-b503-9272a068a7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-f5732e71-db0e-4bb9-be80-c89cea1ace52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:18:56.599682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:56.600553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:18:56.602056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:18:56.602056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:18:56.602056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:56.602056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 231/461] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:56.607086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5732e71-db0e-4bb9-be80-c89cea1ace52 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:56.644402 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2937bfd1-6000-42f3-b503-9272a068a7d6 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:56.651594 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2937bfd1-6000-42f3-b503-9272a068a7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:56.714385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5732e71-db0e-4bb9-be80-c89cea1ace52 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:56.752434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c642e8-0d46-43fa-8f58-e15cfde90201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:56.753994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c642e8-0d46-43fa-8f58-e15cfde90201 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:56.754277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c642e8-0d46-43fa-8f58-e15cfde90201 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:56.754549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c642e8-0d46-43fa-8f58-e15cfde90201 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:56.755198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c642e8-0d46-43fa-8f58-e15cfde90201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:56.755497 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c642e8-0d46-43fa-8f58-e15cfde90201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:56.757699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c642e8-0d46-43fa-8f58-e15cfde90201 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:56.772086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 232/462] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:18:56.772936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c642e8-0d46-43fa-8f58-e15cfde90201 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:56.788888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 231/463] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:56.790263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a58b0-ce99-4472-85b9-dbaa29158fb4 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:56.790556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a58b0-ce99-4472-85b9-dbaa29158fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:56.790829 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a58b0-ce99-4472-85b9-dbaa29158fb4 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:56.795915 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 232/464] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:56.795915 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a58b0-ce99-4472-85b9-dbaa29158fb4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:56.835700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:56.872507 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:56.896672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c642e8-0d46-43fa-8f58-e15cfde90201 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:56.910846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a58b0-ce99-4472-85b9-dbaa29158fb4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:56.933358 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c79457-b07a-4958-aebf-849dcedd0407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:56.934295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c79457-b07a-4958-aebf-849dcedd0407 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:56.934750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c79457-b07a-4958-aebf-849dcedd0407 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:56.935194 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c79457-b07a-4958-aebf-849dcedd0407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:56.946599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c79457-b07a-4958-aebf-849dcedd0407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:56.948780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c79457-b07a-4958-aebf-849dcedd0407 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:56.949437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbd8d2-c70f-4794-b2c2-0fd7a068f635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:56.950110 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbd8d2-c70f-4794-b2c2-0fd7a068f635 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:56.950388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbd8d2-c70f-4794-b2c2-0fd7a068f635 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:56.950659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbd8d2-c70f-4794-b2c2-0fd7a068f635 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:56.951298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbd8d2-c70f-4794-b2c2-0fd7a068f635 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:56.953028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbd8d2-c70f-4794-b2c2-0fd7a068f635 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-12c79457-b07a-4958-aebf-849dcedd0407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:18:56.966438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:56.967498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:18:56.973885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:18:56.973885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:18:56.973885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:56.973885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 233/465] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:56.973885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57295f7-fc61-419d-8442-174bbe7efe66 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:56.973885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57295f7-fc61-419d-8442-174bbe7efe66 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:56.973885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57295f7-fc61-419d-8442-174bbe7efe66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:56.985717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 233/466] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:18:56 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:18:56.986941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbd8d2-c70f-4794-b2c2-0fd7a068f635 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:57.001084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b57295f7-fc61-419d-8442-174bbe7efe66 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:57.001084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b57295f7-fc61-419d-8442-174bbe7efe66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:57.120738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbd8d2-c70f-4794-b2c2-0fd7a068f635 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:57.141447 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 234/467] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:57.142849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57295f7-fc61-419d-8442-174bbe7efe66 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:57.151233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:57.166310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfdb0f-5579-4784-ba26-4a702ef103a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:57.167211 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfdb0f-5579-4784-ba26-4a702ef103a6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:57.167503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfdb0f-5579-4784-ba26-4a702ef103a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:57.167782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfdb0f-5579-4784-ba26-4a702ef103a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:57.168449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfdb0f-5579-4784-ba26-4a702ef103a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:57.168780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfdb0f-5579-4784-ba26-4a702ef103a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:57.170678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfdb0f-5579-4784-ba26-4a702ef103a6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:57.180900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:57.191912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 234/468] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:57.194470 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfdb0f-5579-4784-ba26-4a702ef103a6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:57.271578 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57295f7-fc61-419d-8442-174bbe7efe66 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:57.313831 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfdb0f-5579-4784-ba26-4a702ef103a6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:57.341606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b084384-0d91-4d42-a562-2f77978d747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:57.342775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b084384-0d91-4d42-a562-2f77978d747d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:57.343255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b084384-0d91-4d42-a562-2f77978d747d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:57.343725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b084384-0d91-4d42-a562-2f77978d747d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:57.351415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208f96a-e2c9-41bf-bb52-7135e333578e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:57.352036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208f96a-e2c9-41bf-bb52-7135e333578e None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:57.352319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208f96a-e2c9-41bf-bb52-7135e333578e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:57.352592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208f96a-e2c9-41bf-bb52-7135e333578e None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:57.353550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208f96a-e2c9-41bf-bb52-7135e333578e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:57.354999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208f96a-e2c9-41bf-bb52-7135e333578e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:57.358486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b084384-0d91-4d42-a562-2f77978d747d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:57.360154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b084384-0d91-4d42-a562-2f77978d747d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:57.383362 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 235/469] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:18:57 2022] POST /identity/v3/users => generated 248 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-2b084384-0d91-4d42-a562-2f77978d747d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:18:57.383535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:57.384549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:18:57.385423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:18:57.385423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:18:57.385423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:57.385423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 235/470] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:57.391069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208f96a-e2c9-41bf-bb52-7135e333578e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:57.441950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ecf38-762d-4b83-ad57-78a63d5f9ef9 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:57.442575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ecf38-762d-4b83-ad57-78a63d5f9ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:57.443059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ecf38-762d-4b83-ad57-78a63d5f9ef9 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:57.445035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 236/471] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:18:57.450136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aebc4-f586-41c8-bdc7-f2dbb6a0420f None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:57.450744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aebc4-f586-41c8-bdc7-f2dbb6a0420f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:57.451217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aebc4-f586-41c8-bdc7-f2dbb6a0420f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:57.474542 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:57.485370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-2a5aebc4-f586-41c8-bdc7-f2dbb6a0420f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:57.488859 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2a5aebc4-f586-41c8-bdc7-f2dbb6a0420f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:57.559459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208f96a-e2c9-41bf-bb52-7135e333578e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:57.573527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 237/472] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:18:57.574827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5aebc4-f586-41c8-bdc7-f2dbb6a0420f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:57.585878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:57.611768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26118a6a-6de6-426d-a7ae-8b950696f977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:57.612630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26118a6a-6de6-426d-a7ae-8b950696f977 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:57.613121 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26118a6a-6de6-426d-a7ae-8b950696f977 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:57.613584 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26118a6a-6de6-426d-a7ae-8b950696f977 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:57.614522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26118a6a-6de6-426d-a7ae-8b950696f977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:57.614969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26118a6a-6de6-426d-a7ae-8b950696f977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:57.616826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26118a6a-6de6-426d-a7ae-8b950696f977 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:57.632300 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 236/473] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:57.639141 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26118a6a-6de6-426d-a7ae-8b950696f977 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:57.703791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5aebc4-f586-41c8-bdc7-f2dbb6a0420f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:57.757979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad75254-7f0a-4e06-895f-b56b20dcd30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:57.758890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75254-7f0a-4e06-895f-b56b20dcd30a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:57.759175 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75254-7f0a-4e06-895f-b56b20dcd30a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:57.759613 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad75254-7f0a-4e06-895f-b56b20dcd30a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:57.765771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:57.772203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26118a6a-6de6-426d-a7ae-8b950696f977 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:57.776287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad75254-7f0a-4e06-895f-b56b20dcd30a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:57.777779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad75254-7f0a-4e06-895f-b56b20dcd30a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:57.809745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dc51a-17f7-4011-9309-607231818d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-4ad75254-7f0a-4e06-895f-b56b20dcd30a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:18:57.809987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:57.811003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:18:57.811964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:18:57.811964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:18:57.811964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:18:57.811964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 238/474] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:18:57.812920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dc51a-17f7-4011-9309-607231818d90 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:57.813341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dc51a-17f7-4011-9309-607231818d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:57.813752 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dc51a-17f7-4011-9309-607231818d90 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:57.814535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65dc51a-17f7-4011-9309-607231818d90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:57.816120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65dc51a-17f7-4011-9309-607231818d90 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:57.820609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad75254-7f0a-4e06-895f-b56b20dcd30a None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:57.843141 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 237/475] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:18:57 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:18:57.887215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dc51a-17f7-4011-9309-607231818d90 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:57.932515 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:58.011425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dc51a-17f7-4011-9309-607231818d90 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:58.022553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad75254-7f0a-4e06-895f-b56b20dcd30a None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:58.055683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fc880-924a-4227-935b-1602c89f309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:58.056912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fc880-924a-4227-935b-1602c89f309a None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:58.057407 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fc880-924a-4227-935b-1602c89f309a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:58.057889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fc880-924a-4227-935b-1602c89f309a None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:58.058796 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fc880-924a-4227-935b-1602c89f309a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:58.060484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fc880-924a-4227-935b-1602c89f309a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:58.071668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.sql.core [None req-a36fc880-924a-4227-935b-1602c89f309a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 19 16:18:58.071668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 19 16:18:58.071668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [parameters: {'id': '23dedebef52e4f068365af576d1338aa', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 19 16:18:58.071668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81551) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a36fc880-924a-4227-935b-1602c89f309a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application conn.query(q) Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result.read() Aug 19 16:18:58.090502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return ref.to_dict() Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 16:18:58.091474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application next(self.gen) Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application yield resource Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application next(self.gen) Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.session.rollback() Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.force_reraise() Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise self.value Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application session.commit() Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 19 16:18:58.092358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._prepare_impl() Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.session.flush() Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._flush(objects) Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application compat.raise_( Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application flush_context.execute() Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rec.execute(self) Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 19 16:18:58.093234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 16:18:58.094865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application conn.query(q) Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result.read() Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 16:18:58.095870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [parameters: {'id': '23dedebef52e4f068365af576d1338aa', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:58.096634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 19 16:18:58.097433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:58.099220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 238/476] 10.222.0.55 () {60 vars in 1131 bytes} [Fri Aug 19 16:18:57 2022] POST /identity/v3/roles => generated 144 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:18:58.100484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fc880-924a-4227-935b-1602c89f309a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:58.103475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fc895-3fce-4e80-aace-2457d3f24f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:58.103720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fc895-3fce-4e80-aace-2457d3f24f57 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:58.103917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fc895-3fce-4e80-aace-2457d3f24f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:58.104312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fc895-3fce-4e80-aace-2457d3f24f57 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:58.104980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fc895-3fce-4e80-aace-2457d3f24f57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:58.105278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fc895-3fce-4e80-aace-2457d3f24f57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:58.106646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fc895-3fce-4e80-aace-2457d3f24f57 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:58.126340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 239/477] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:58.127256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fc895-3fce-4e80-aace-2457d3f24f57 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:58.130367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:58.236561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fc880-924a-4227-935b-1602c89f309a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:58.244716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fc895-3fce-4e80-aace-2457d3f24f57 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:58.269523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:58.283671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bb0c4-866a-4c76-8d73-1b90ddb3ca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:58.284581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bb0c4-866a-4c76-8d73-1b90ddb3ca27 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:58.284900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bb0c4-866a-4c76-8d73-1b90ddb3ca27 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:58.285178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bb0c4-866a-4c76-8d73-1b90ddb3ca27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:58.289150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad536ee-0087-406c-bdb0-16cd6ea90a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:58.289764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad536ee-0087-406c-bdb0-16cd6ea90a91 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:58.290046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad536ee-0087-406c-bdb0-16cd6ea90a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:58.290321 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad536ee-0087-406c-bdb0-16cd6ea90a91 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:58.290965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad536ee-0087-406c-bdb0-16cd6ea90a91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:58.294122 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad536ee-0087-406c-bdb0-16cd6ea90a91 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:58.297818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bb0c4-866a-4c76-8d73-1b90ddb3ca27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-5e2bb0c4-866a-4c76-8d73-1b90ddb3ca27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:18:58.303464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:18:58.304349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:18:58.305158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:18:58.305158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:18:58.305158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:18:58.305158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:18:58.305158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:18:58.305158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:18:58.305158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:18:58.305158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:18:58.305158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:18:58.305158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:18:58.305158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 16:18:58.305158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:18:58.307040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 239/478] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:18:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:18:58.312374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bb0c4-866a-4c76-8d73-1b90ddb3ca27 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:58.324856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 240/479] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:18:58 2022] POST /identity/v3/users => generated 249 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:18:58.417819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:18:58.435639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bb0c4-866a-4c76-8d73-1b90ddb3ca27 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:18:58.473935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf8f71-f1b3-4997-a8cf-1bd4f55f347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:18:58.475156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf8f71-f1b3-4997-a8cf-1bd4f55f347f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:58.475579 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf8f71-f1b3-4997-a8cf-1bd4f55f347f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:58.476037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf8f71-f1b3-4997-a8cf-1bd4f55f347f None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:58.476780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf8f71-f1b3-4997-a8cf-1bd4f55f347f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:18:58.477108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf8f71-f1b3-4997-a8cf-1bd4f55f347f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:18:58.478835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf8f71-f1b3-4997-a8cf-1bd4f55f347f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:18:58.489256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 240/480] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:18:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:18:59.370456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3bd1f-b7dc-4181-bc56-afe638d44f08 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:59.370992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3bd1f-b7dc-4181-bc56-afe638d44f08 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:59.370992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3bd1f-b7dc-4181-bc56-afe638d44f08 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:59.372614 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 241/481] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:59.377531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2e6e8-a2e6-4e6a-a210-455c14a8cf29 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:59.377922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2e6e8-a2e6-4e6a-a210-455c14a8cf29 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:59.378083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2e6e8-a2e6-4e6a-a210-455c14a8cf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:59.405755 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-fec2e6e8-a2e6-4e6a-a210-455c14a8cf29 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:59.407327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fec2e6e8-a2e6-4e6a-a210-455c14a8cf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:59.480582 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 241/482] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:18:59.736782 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62006635-da14-4e6a-ba44-61db75d61f20 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:59.737099 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62006635-da14-4e6a-ba44-61db75d61f20 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:59.737326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62006635-da14-4e6a-ba44-61db75d61f20 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:59.738779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 242/483] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:18:59.750179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6fdac-7846-48c1-945a-e4eab8ec00cb None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:18:59.750904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6fdac-7846-48c1-945a-e4eab8ec00cb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:18:59.751582 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6fdac-7846-48c1-945a-e4eab8ec00cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:18:59.781608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-34d6fdac-7846-48c1-945a-e4eab8ec00cb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:18:59.783222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-34d6fdac-7846-48c1-945a-e4eab8ec00cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:18:59.861573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62006635-da14-4e6a-ba44-61db75d61f20 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:18:59.867684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 242/484] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:00.018582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62006635-da14-4e6a-ba44-61db75d61f20 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:00.067113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789563b-d46b-4fd4-b8b6-4ba1d4b1f1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:00.067775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789563b-d46b-4fd4-b8b6-4ba1d4b1f1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:00.068060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789563b-d46b-4fd4-b8b6-4ba1d4b1f1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:00.068918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789563b-d46b-4fd4-b8b6-4ba1d4b1f1b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:00.088688 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789563b-d46b-4fd4-b8b6-4ba1d4b1f1b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-e789563b-d46b-4fd4-b8b6-4ba1d4b1f1b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:00.093629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:00.094474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:00.095251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 243/485] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:18:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:00.101977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6fdac-7846-48c1-945a-e4eab8ec00cb None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:00.117106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:00.217733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789563b-d46b-4fd4-b8b6-4ba1d4b1f1b3 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:00.237083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6fdac-7846-48c1-945a-e4eab8ec00cb None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:00.265234 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:00.326892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803feda2-75d8-4784-b48f-77e2aeb3659d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:00.327433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803feda2-75d8-4784-b48f-77e2aeb3659d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:00.327710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803feda2-75d8-4784-b48f-77e2aeb3659d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:00.327978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803feda2-75d8-4784-b48f-77e2aeb3659d None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:00.328654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803feda2-75d8-4784-b48f-77e2aeb3659d None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:00.329002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803feda2-75d8-4784-b48f-77e2aeb3659d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:00.330372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803feda2-75d8-4784-b48f-77e2aeb3659d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:00.335499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789563b-d46b-4fd4-b8b6-4ba1d4b1f1b3 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:00.353147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 243/486] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:00.361706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803feda2-75d8-4784-b48f-77e2aeb3659d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:00.374946 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d56f00-c55d-4ffb-902f-57d42b1ea0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:00.375577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d56f00-c55d-4ffb-902f-57d42b1ea0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:00.375865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d56f00-c55d-4ffb-902f-57d42b1ea0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:00.376144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d56f00-c55d-4ffb-902f-57d42b1ea0a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:00.388086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d56f00-c55d-4ffb-902f-57d42b1ea0a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-61d56f00-c55d-4ffb-902f-57d42b1ea0a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:00.394137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:00.394985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:00.395764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 244/487] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:00.401766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d56f00-c55d-4ffb-902f-57d42b1ea0a7 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:00.512869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d56f00-c55d-4ffb-902f-57d42b1ea0a7 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:00.520254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:00.525598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:00.553445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b181724-d355-4853-afa8-93232fe22c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:00.554424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b181724-d355-4853-afa8-93232fe22c06 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:00.554847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b181724-d355-4853-afa8-93232fe22c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:00.555264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b181724-d355-4853-afa8-93232fe22c06 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:00.556109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b181724-d355-4853-afa8-93232fe22c06 None admin] RBAC: Adding query filter params (name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:00.556556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b181724-d355-4853-afa8-93232fe22c06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:00.558120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b181724-d355-4853-afa8-93232fe22c06 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:00.567616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 245/488] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:00.573439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b181724-d355-4853-afa8-93232fe22c06 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:00.598731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803feda2-75d8-4784-b48f-77e2aeb3659d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:00.666493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795b162-a8e4-4b23-a525-e2fdaa548095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:00.667337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795b162-a8e4-4b23-a525-e2fdaa548095 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:00.667788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795b162-a8e4-4b23-a525-e2fdaa548095 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:00.668213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795b162-a8e4-4b23-a525-e2fdaa548095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:00.680045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795b162-a8e4-4b23-a525-e2fdaa548095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:00.681718 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795b162-a8e4-4b23-a525-e2fdaa548095 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:00.682935 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b181724-d355-4853-afa8-93232fe22c06 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c795b162-a8e4-4b23-a525-e2fdaa548095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:00.697135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:00.698244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:00.699066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:00.699066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:00.699066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:00.705280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 244/489] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:00.706906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e97580-00f7-4ee2-beeb-91dc96cc63bc None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:00.707352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e97580-00f7-4ee2-beeb-91dc96cc63bc None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:00.707815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e97580-00f7-4ee2-beeb-91dc96cc63bc None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:00.710282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 245/490] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:00.711337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e97580-00f7-4ee2-beeb-91dc96cc63bc None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:00.720146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b92c1-4506-427f-acf9-23406ae1b974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:00.720815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b92c1-4506-427f-acf9-23406ae1b974 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:00.721104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b92c1-4506-427f-acf9-23406ae1b974 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:00.721378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b92c1-4506-427f-acf9-23406ae1b974 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:00.735931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b92c1-4506-427f-acf9-23406ae1b974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:00.737741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b92c1-4506-427f-acf9-23406ae1b974 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-984b92c1-4506-427f-acf9-23406ae1b974 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:00.753513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:00.754460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:00.755775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:00.755775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:00.755775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:00.755775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 246/491] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:00.756301 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fa4f9-88ad-4dbb-b437-1dc4139bcc7c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:00.756458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fa4f9-88ad-4dbb-b437-1dc4139bcc7c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:00.761584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fa4f9-88ad-4dbb-b437-1dc4139bcc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:00.781263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-dc3fa4f9-88ad-4dbb-b437-1dc4139bcc7c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:00.782473 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-dc3fa4f9-88ad-4dbb-b437-1dc4139bcc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:00.786429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:00.828143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e97580-00f7-4ee2-beeb-91dc96cc63bc None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:00.848313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:00.859473 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 247/492] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:00.860146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fa4f9-88ad-4dbb-b437-1dc4139bcc7c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:00.876572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844c05e-712e-4aad-9be9-a0990cc5d60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:00.877862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844c05e-712e-4aad-9be9-a0990cc5d60b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:00.878740 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844c05e-712e-4aad-9be9-a0990cc5d60b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:00.879263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844c05e-712e-4aad-9be9-a0990cc5d60b None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:00.880147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844c05e-712e-4aad-9be9-a0990cc5d60b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:00.881189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844c05e-712e-4aad-9be9-a0990cc5d60b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:00.883709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844c05e-712e-4aad-9be9-a0990cc5d60b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:00.902301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 246/493] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:00.903940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d6b96-347e-4a28-b305-024b732672ab None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:00.904523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d6b96-347e-4a28-b305-024b732672ab None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:00.905019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d6b96-347e-4a28-b305-024b732672ab None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:00.910667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 247/494] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:00.910667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d6b96-347e-4a28-b305-024b732672ab None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:00.989939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fa4f9-88ad-4dbb-b437-1dc4139bcc7c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:01.020118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d6b96-347e-4a28-b305-024b732672ab None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:01.029296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eadfc1-907c-4caf-9b02-e6fc2f278857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:01.030289 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eadfc1-907c-4caf-9b02-e6fc2f278857 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.030766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eadfc1-907c-4caf-9b02-e6fc2f278857 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.031231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eadfc1-907c-4caf-9b02-e6fc2f278857 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.032182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eadfc1-907c-4caf-9b02-e6fc2f278857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:01.032675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eadfc1-907c-4caf-9b02-e6fc2f278857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:01.034364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eadfc1-907c-4caf-9b02-e6fc2f278857 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:01.043689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:01.048503 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 248/495] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:01.050370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25668a1-6aaf-46b5-a01e-0d60cbc4357d None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.051652 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25668a1-6aaf-46b5-a01e-0d60cbc4357d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.052127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25668a1-6aaf-46b5-a01e-0d60cbc4357d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.056441 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a48c57-e9a7-45f6-a417-64e9cbada661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:01.057106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a48c57-e9a7-45f6-a417-64e9cbada661 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.057385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a48c57-e9a7-45f6-a417-64e9cbada661 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.057662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a48c57-e9a7-45f6-a417-64e9cbada661 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.065511 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a48c57-e9a7-45f6-a417-64e9cbada661 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:01.067040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a48c57-e9a7-45f6-a417-64e9cbada661 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:01.073357 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f25668a1-6aaf-46b5-a01e-0d60cbc4357d None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:01.074824 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f25668a1-6aaf-46b5-a01e-0d60cbc4357d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-78a48c57-e9a7-45f6-a417-64e9cbada661 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:01.081357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:01.082223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:01.083113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:01.083113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:01.083113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:01.083113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:01.083113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:01.083113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:01.083113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:01.083113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:01.083113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:19:01.083113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:01.087772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 248/496] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Aug 19 16:19:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:01.088694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a48c57-e9a7-45f6-a417-64e9cbada661 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:01.127512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:01.155229 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 249/497] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:01.155948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25668a1-6aaf-46b5-a01e-0d60cbc4357d None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:01.215631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a48c57-e9a7-45f6-a417-64e9cbada661 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:01.250439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0469a-02cb-4fde-b8df-f5d4e5f6f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:01.251196 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0469a-02cb-4fde-b8df-f5d4e5f6f064 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.251479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0469a-02cb-4fde-b8df-f5d4e5f6f064 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.251753 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0469a-02cb-4fde-b8df-f5d4e5f6f064 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.259810 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0469a-02cb-4fde-b8df-f5d4e5f6f064 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:01.261291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0469a-02cb-4fde-b8df-f5d4e5f6f064 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:01.264740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25668a1-6aaf-46b5-a01e-0d60cbc4357d None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a7d0469a-02cb-4fde-b8df-f5d4e5f6f064 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:01.272188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:01.273095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:01.275095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:01.275095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:01.275095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:01.275095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:01.275095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:01.275095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:01.275095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:01.275095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:01.275095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 19 16:19:01.275095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:01.275095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 249/498] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:01.275095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0469a-02cb-4fde-b8df-f5d4e5f6f064 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:01.298529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:01.305661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518d690-5fa8-4817-aba8-8d1d5caa5b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:01.306278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518d690-5fa8-4817-aba8-8d1d5caa5b45 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.306558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518d690-5fa8-4817-aba8-8d1d5caa5b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.306831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518d690-5fa8-4817-aba8-8d1d5caa5b45 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.307546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518d690-5fa8-4817-aba8-8d1d5caa5b45 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:01.307856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518d690-5fa8-4817-aba8-8d1d5caa5b45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:01.309340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518d690-5fa8-4817-aba8-8d1d5caa5b45 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:01.351546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 250/499] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Aug 19 16:19:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:01.353432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5f7c4-696d-40ff-a339-7e9d3aea1d07 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.353963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5f7c4-696d-40ff-a339-7e9d3aea1d07 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.354442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5f7c4-696d-40ff-a339-7e9d3aea1d07 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.356239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 251/500] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:01.357167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5f7c4-696d-40ff-a339-7e9d3aea1d07 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:01.388972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:01.394997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0469a-02cb-4fde-b8df-f5d4e5f6f064 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:01.429509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83025eaf-1c84-44fb-bdcf-81f8f06d45ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:01.430135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83025eaf-1c84-44fb-bdcf-81f8f06d45ea None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.430412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83025eaf-1c84-44fb-bdcf-81f8f06d45ea None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.430678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83025eaf-1c84-44fb-bdcf-81f8f06d45ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.440476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83025eaf-1c84-44fb-bdcf-81f8f06d45ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-83025eaf-1c84-44fb-bdcf-81f8f06d45ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:01.446174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:01.447228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 250/501] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:01.448182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83025eaf-1c84-44fb-bdcf-81f8f06d45ea None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:01.473317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5f7c4-696d-40ff-a339-7e9d3aea1d07 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:01.508694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c42e2-9459-4994-b040-e2d50a95fbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:01.509339 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c42e2-9459-4994-b040-e2d50a95fbdc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.509625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c42e2-9459-4994-b040-e2d50a95fbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.509898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c42e2-9459-4994-b040-e2d50a95fbdc None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.510616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c42e2-9459-4994-b040-e2d50a95fbdc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:01.510920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c42e2-9459-4994-b040-e2d50a95fbdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:01.512404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c42e2-9459-4994-b040-e2d50a95fbdc None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:01.529365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 252/502] 10.222.0.55 () {56 vars in 1137 bytes} [Fri Aug 19 16:19:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:01.530547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77712a3a-8f6b-4d79-818e-9a9838f3e5e8 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.530836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77712a3a-8f6b-4d79-818e-9a9838f3e5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.531116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77712a3a-8f6b-4d79-818e-9a9838f3e5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.547669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83025eaf-1c84-44fb-bdcf-81f8f06d45ea None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:01.548243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-77712a3a-8f6b-4d79-818e-9a9838f3e5e8 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:01.549482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-77712a3a-8f6b-4d79-818e-9a9838f3e5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:01.581497 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebf2fc-7cd7-4f3b-b7bb-907b8495ee3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:01.582144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebf2fc-7cd7-4f3b-b7bb-907b8495ee3e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.582421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebf2fc-7cd7-4f3b-b7bb-907b8495ee3e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.582693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebf2fc-7cd7-4f3b-b7bb-907b8495ee3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.591895 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:01.608507 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:01.617864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 253/503] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:01.618676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77712a3a-8f6b-4d79-818e-9a9838f3e5e8 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:01.669143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebf2fc-7cd7-4f3b-b7bb-907b8495ee3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:01.670745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebf2fc-7cd7-4f3b-b7bb-907b8495ee3e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:01.731580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77712a3a-8f6b-4d79-818e-9a9838f3e5e8 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:01.767845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c2bd8-58f3-4992-b25f-1c6d6b57a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:01.768567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c2bd8-58f3-4992-b25f-1c6d6b57a204 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.768888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c2bd8-58f3-4992-b25f-1c6d6b57a204 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.769044 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c2bd8-58f3-4992-b25f-1c6d6b57a204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.776507 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 251/504] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:19:01.779418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c2bd8-58f3-4992-b25f-1c6d6b57a204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:01.780960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c2bd8-58f3-4992-b25f-1c6d6b57a204 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:01.781599 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebf2fc-7cd7-4f3b-b7bb-907b8495ee3e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-197c2bd8-58f3-4992-b25f-1c6d6b57a204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:01.798030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:01.798983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:01.799796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:01.799796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:01.799796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:01.800355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 254/505] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:01.800965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c2bd8-58f3-4992-b25f-1c6d6b57a204 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:01.829809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:01.845380 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:01.893040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebf2fc-7cd7-4f3b-b7bb-907b8495ee3e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:01.903827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c2bd8-58f3-4992-b25f-1c6d6b57a204 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:01.928709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9aae3-6262-48a6-b48b-1488d0d62430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:01.929274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9aae3-6262-48a6-b48b-1488d0d62430 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.929557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9aae3-6262-48a6-b48b-1488d0d62430 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.929832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9aae3-6262-48a6-b48b-1488d0d62430 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.930540 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9aae3-6262-48a6-b48b-1488d0d62430 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:01.930845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9aae3-6262-48a6-b48b-1488d0d62430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:01.932221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9aae3-6262-48a6-b48b-1488d0d62430 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:01.938156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ff925-92ae-4709-948f-3850706e0b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:01.938747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ff925-92ae-4709-948f-3850706e0b70 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:01.939021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ff925-92ae-4709-948f-3850706e0b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:01.939290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ff925-92ae-4709-948f-3850706e0b70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:01.939944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 252/506] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:01.940725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9aae3-6262-48a6-b48b-1488d0d62430 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.007029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ff925-92ae-4709-948f-3850706e0b70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:02.008611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ff925-92ae-4709-948f-3850706e0b70 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:02.035223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:02.038627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9aae3-6262-48a6-b48b-1488d0d62430 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:02.055555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:02.073166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43e0d1-fbc5-48cc-b450-f4471aaa4dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:02.073794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43e0d1-fbc5-48cc-b450-f4471aaa4dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:02.074076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43e0d1-fbc5-48cc-b450-f4471aaa4dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:02.074351 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43e0d1-fbc5-48cc-b450-f4471aaa4dd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:02.085239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43e0d1-fbc5-48cc-b450-f4471aaa4dd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:02.086683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43e0d1-fbc5-48cc-b450-f4471aaa4dd3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-3a43e0d1-fbc5-48cc-b450-f4471aaa4dd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:02.101277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:02.102334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:02.103505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:02.103505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:02.103505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:02.103505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 253/507] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:02.103505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43e0d1-fbc5-48cc-b450-f4471aaa4dd3 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.109544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 255/508] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:19:02.110373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ff925-92ae-4709-948f-3850706e0b70 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.202076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43e0d1-fbc5-48cc-b450-f4471aaa4dd3 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:02.207384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ff925-92ae-4709-948f-3850706e0b70 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:02.234608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:02.235055 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541c33f-294d-45d7-9ee6-118ee7e15c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:02.235716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541c33f-294d-45d7-9ee6-118ee7e15c7e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:02.235969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541c33f-294d-45d7-9ee6-118ee7e15c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:02.236302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541c33f-294d-45d7-9ee6-118ee7e15c7e None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:02.236982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541c33f-294d-45d7-9ee6-118ee7e15c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:02.237286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541c33f-294d-45d7-9ee6-118ee7e15c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:02.238663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541c33f-294d-45d7-9ee6-118ee7e15c7e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:02.243164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdc0a7-d4a6-4761-8c21-d23bfa84f178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:02.243780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdc0a7-d4a6-4761-8c21-d23bfa84f178 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:02.244055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdc0a7-d4a6-4761-8c21-d23bfa84f178 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:02.244322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdc0a7-d4a6-4761-8c21-d23bfa84f178 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:02.250488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 254/509] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:02.251273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541c33f-294d-45d7-9ee6-118ee7e15c7e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.259234 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdc0a7-d4a6-4761-8c21-d23bfa84f178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-e1cdc0a7-d4a6-4761-8c21-d23bfa84f178 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:02.264031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:02.265140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 256/510] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:02.270797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdc0a7-d4a6-4761-8c21-d23bfa84f178 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.281002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:02.350499 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541c33f-294d-45d7-9ee6-118ee7e15c7e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:02.364495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdc0a7-d4a6-4761-8c21-d23bfa84f178 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:02.384485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e5cfb-d8f7-4eca-ad40-e52d317c6bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:02.385149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e5cfb-d8f7-4eca-ad40-e52d317c6bda None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:02.385431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e5cfb-d8f7-4eca-ad40-e52d317c6bda None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:02.385707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e5cfb-d8f7-4eca-ad40-e52d317c6bda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:02.395626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e5cfb-d8f7-4eca-ad40-e52d317c6bda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:02.397648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e5cfb-d8f7-4eca-ad40-e52d317c6bda None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:02.400065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214185ba-6376-46f0-b4ba-174c2f53ddd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:02.400658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214185ba-6376-46f0-b4ba-174c2f53ddd3 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:02.400961 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214185ba-6376-46f0-b4ba-174c2f53ddd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:02.401229 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214185ba-6376-46f0-b4ba-174c2f53ddd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:02.401855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214185ba-6376-46f0-b4ba-174c2f53ddd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:02.402152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214185ba-6376-46f0-b4ba-174c2f53ddd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:02.403499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214185ba-6376-46f0-b4ba-174c2f53ddd3 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-0d8e5cfb-d8f7-4eca-ad40-e52d317c6bda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:02.412480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:02.413536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:02.414459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:02.414459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:02.414459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:02.414459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 255/511] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:02.414459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e5cfb-d8f7-4eca-ad40-e52d317c6bda None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.421785 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 257/512] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:02.421785 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214185ba-6376-46f0-b4ba-174c2f53ddd3 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.472019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:02.499738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:02.519499 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e5cfb-d8f7-4eca-ad40-e52d317c6bda None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:02.525548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214185ba-6376-46f0-b4ba-174c2f53ddd3 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:02.553207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8e97a-c294-471a-bb4e-b8bce709ad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:02.553832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8e97a-c294-471a-bb4e-b8bce709ad98 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:02.554107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8e97a-c294-471a-bb4e-b8bce709ad98 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:02.554378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8e97a-c294-471a-bb4e-b8bce709ad98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:02.557316 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc6751-a058-46fa-9d88-f890a7de93a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:02.564658 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8e97a-c294-471a-bb4e-b8bce709ad98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:02.565167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc6751-a058-46fa-9d88-f890a7de93a0 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:02.565499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc6751-a058-46fa-9d88-f890a7de93a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:02.566170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8e97a-c294-471a-bb4e-b8bce709ad98 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:02.566270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc6751-a058-46fa-9d88-f890a7de93a0 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:02.566895 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc6751-a058-46fa-9d88-f890a7de93a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:02.568903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc6751-a058-46fa-9d88-f890a7de93a0 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-60e8e97a-c294-471a-bb4e-b8bce709ad98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:02.580679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:02.582079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:02.583074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:02.583074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:02.583074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:02.583074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 256/513] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:02.583340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8e97a-c294-471a-bb4e-b8bce709ad98 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.594998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 258/514] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:19:02 2022] POST /identity/v3/users => generated 247 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:02.595798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc6751-a058-46fa-9d88-f890a7de93a0 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.705211 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8e97a-c294-471a-bb4e-b8bce709ad98 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:02.723264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc6751-a058-46fa-9d88-f890a7de93a0 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:02.728627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:02.747024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436de92b-acdf-4942-9d89-73d26ed47d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:02.747644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436de92b-acdf-4942-9d89-73d26ed47d34 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:02.747929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436de92b-acdf-4942-9d89-73d26ed47d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:02.748205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436de92b-acdf-4942-9d89-73d26ed47d34 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:02.748920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436de92b-acdf-4942-9d89-73d26ed47d34 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:02.749229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436de92b-acdf-4942-9d89-73d26ed47d34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:02.750616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436de92b-acdf-4942-9d89-73d26ed47d34 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:02.752084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:02.762083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 257/515] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:02.762902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436de92b-acdf-4942-9d89-73d26ed47d34 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.769756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8122c5-95ed-4bed-bafe-d249fba721c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:02.770274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8122c5-95ed-4bed-bafe-d249fba721c2 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:02.770553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8122c5-95ed-4bed-bafe-d249fba721c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:02.770825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8122c5-95ed-4bed-bafe-d249fba721c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:02.771469 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8122c5-95ed-4bed-bafe-d249fba721c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:02.771770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8122c5-95ed-4bed-bafe-d249fba721c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:02.773202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8122c5-95ed-4bed-bafe-d249fba721c2 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:02.791038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 259/516] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:02.791901 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8122c5-95ed-4bed-bafe-d249fba721c2 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.879353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436de92b-acdf-4942-9d89-73d26ed47d34 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:02.902629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8122c5-95ed-4bed-bafe-d249fba721c2 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:02.916330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b89ca7-ee22-4e61-8939-5ceb902d75f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:02.916993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89ca7-ee22-4e61-8939-5ceb902d75f5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:02.917276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89ca7-ee22-4e61-8939-5ceb902d75f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:02.917837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89ca7-ee22-4e61-8939-5ceb902d75f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:02.924495 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b89ca7-ee22-4e61-8939-5ceb902d75f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:02.925974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b89ca7-ee22-4e61-8939-5ceb902d75f5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-14b89ca7-ee22-4e61-8939-5ceb902d75f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:02.939060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:02.939917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:02.940779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:02.940779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:02.940779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:02.940779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:02.940779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:02.940779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:02.940779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:02.940779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:02.941207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ed8c1-b7ad-4102-b75c-7bce5f5a2809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:02.941207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ed8c1-b7ad-4102-b75c-7bce5f5a2809 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:02.941207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ed8c1-b7ad-4102-b75c-7bce5f5a2809 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:02.941410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 258/517] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:02.941410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b89ca7-ee22-4e61-8939-5ceb902d75f5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.945713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ed8c1-b7ad-4102-b75c-7bce5f5a2809 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:02.946397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ed8c1-b7ad-4102-b75c-7bce5f5a2809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:02.946707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ed8c1-b7ad-4102-b75c-7bce5f5a2809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:02.948097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ed8c1-b7ad-4102-b75c-7bce5f5a2809 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:02.960319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 260/518] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:02.961164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ed8c1-b7ad-4102-b75c-7bce5f5a2809 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:02.991074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:03.009148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:03.062574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b89ca7-ee22-4e61-8939-5ceb902d75f5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:03.074875 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ed8c1-b7ad-4102-b75c-7bce5f5a2809 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:03.103557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499be537-b792-4dff-81a3-9181ed68547c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:03.104183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499be537-b792-4dff-81a3-9181ed68547c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:03.104464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499be537-b792-4dff-81a3-9181ed68547c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:03.104729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499be537-b792-4dff-81a3-9181ed68547c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:03.110531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845201a1-bfd8-4f37-8601-4838bfe49a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:03.111566 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845201a1-bfd8-4f37-8601-4838bfe49a62 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:03.111850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845201a1-bfd8-4f37-8601-4838bfe49a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:03.112118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845201a1-bfd8-4f37-8601-4838bfe49a62 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:03.115974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499be537-b792-4dff-81a3-9181ed68547c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:03.117485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499be537-b792-4dff-81a3-9181ed68547c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:03.121061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845201a1-bfd8-4f37-8601-4838bfe49a62 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:03.122495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845201a1-bfd8-4f37-8601-4838bfe49a62 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-499be537-b792-4dff-81a3-9181ed68547c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:03.131990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:03.132921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:03.140586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:03.140586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:03.140586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:03.140586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 259/519] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:03.140586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499be537-b792-4dff-81a3-9181ed68547c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-845201a1-bfd8-4f37-8601-4838bfe49a62 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:03.140890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:03.141692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:03.142554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:03.142554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:03.142554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:03.142554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:03.142554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:03.142554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:03.142554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:03.142554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:03.142554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 16:19:03.142554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:03.142554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 261/520] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:03.142554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845201a1-bfd8-4f37-8601-4838bfe49a62 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:03.232282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:03.249559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845201a1-bfd8-4f37-8601-4838bfe49a62 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:03.250095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499be537-b792-4dff-81a3-9181ed68547c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:03.260263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:03.285632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126715c7-0dca-443c-99fa-1c6f320789df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:03.286269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126715c7-0dca-443c-99fa-1c6f320789df None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:03.286610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126715c7-0dca-443c-99fa-1c6f320789df None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:03.287418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126715c7-0dca-443c-99fa-1c6f320789df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:03.287951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa826c-2c14-4fd5-a28a-87e0af170a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:03.288739 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa826c-2c14-4fd5-a28a-87e0af170a99 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:03.289053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa826c-2c14-4fd5-a28a-87e0af170a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:03.289432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa826c-2c14-4fd5-a28a-87e0af170a99 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:03.290186 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa826c-2c14-4fd5-a28a-87e0af170a99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:03.290493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa826c-2c14-4fd5-a28a-87e0af170a99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:03.291977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa826c-2c14-4fd5-a28a-87e0af170a99 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:03.294914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126715c7-0dca-443c-99fa-1c6f320789df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:03.296344 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126715c7-0dca-443c-99fa-1c6f320789df None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-126715c7-0dca-443c-99fa-1c6f320789df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:03.309265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:03.310076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:03.310896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:03.310896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:03.310896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:03.310896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:03.310896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:03.310896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:03.310896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:03.310896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:03.310896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 262/521] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:03.311861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126715c7-0dca-443c-99fa-1c6f320789df None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:03.312438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 260/522] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:03.313505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa826c-2c14-4fd5-a28a-87e0af170a99 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:03.418896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa826c-2c14-4fd5-a28a-87e0af170a99 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:03.421430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126715c7-0dca-443c-99fa-1c6f320789df None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:03.454397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f3a1c-78b8-4dea-9f2e-a0f3ff9112c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:03.454996 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f3a1c-78b8-4dea-9f2e-a0f3ff9112c7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:03.455270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f3a1c-78b8-4dea-9f2e-a0f3ff9112c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:03.455540 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f3a1c-78b8-4dea-9f2e-a0f3ff9112c7 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:03.456254 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f3a1c-78b8-4dea-9f2e-a0f3ff9112c7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:03.456553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f3a1c-78b8-4dea-9f2e-a0f3ff9112c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:03.457719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e1cab-61bf-41f8-87d3-fd4fa9b3d538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:03.458026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f3a1c-78b8-4dea-9f2e-a0f3ff9112c7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:03.458319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e1cab-61bf-41f8-87d3-fd4fa9b3d538 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:03.459339 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e1cab-61bf-41f8-87d3-fd4fa9b3d538 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:03.459442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e1cab-61bf-41f8-87d3-fd4fa9b3d538 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:03.460011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e1cab-61bf-41f8-87d3-fd4fa9b3d538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:03.460307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e1cab-61bf-41f8-87d3-fd4fa9b3d538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:03.461681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e1cab-61bf-41f8-87d3-fd4fa9b3d538 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:03.479926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 263/523] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:03.479926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e1cab-61bf-41f8-87d3-fd4fa9b3d538 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:03.482900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:03.505037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 261/524] 10.222.0.55 () {56 vars in 1137 bytes} [Fri Aug 19 16:19:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:03.505037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f3a1c-78b8-4dea-9f2e-a0f3ff9112c7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:03.531866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:03.592320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e1cab-61bf-41f8-87d3-fd4fa9b3d538 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:03.606875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f3a1c-78b8-4dea-9f2e-a0f3ff9112c7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:03.628962 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5618f78-e3a0-4582-aa6c-6251c3b41d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:03.629573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5618f78-e3a0-4582-aa6c-6251c3b41d00 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:03.629854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5618f78-e3a0-4582-aa6c-6251c3b41d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:03.630127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5618f78-e3a0-4582-aa6c-6251c3b41d00 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:03.630863 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5618f78-e3a0-4582-aa6c-6251c3b41d00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:03.631170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5618f78-e3a0-4582-aa6c-6251c3b41d00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:03.632608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5618f78-e3a0-4582-aa6c-6251c3b41d00 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:03.644472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1de5a6-4788-4c7e-a245-74f9b8a86d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:03.645102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1de5a6-4788-4c7e-a245-74f9b8a86d55 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:03.645381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1de5a6-4788-4c7e-a245-74f9b8a86d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:03.645652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1de5a6-4788-4c7e-a245-74f9b8a86d55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:03.647063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1de5a6-4788-4c7e-a245-74f9b8a86d55 None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=4739e80619a54b178b9aacec6a42a7f0) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:03.647380 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1de5a6-4788-4c7e-a245-74f9b8a86d55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:03.649060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1de5a6-4788-4c7e-a245-74f9b8a86d55 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:03.656667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 264/525] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:03.656667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5618f78-e3a0-4582-aa6c-6251c3b41d00 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:03.673206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 262/526] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:03 2022] GET /identity/v3/role_assignments?user.id=4739e80619a54b178b9aacec6a42a7f0&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:03.673206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1de5a6-4788-4c7e-a245-74f9b8a86d55 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:03.732043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:03.774449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:03.778092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5618f78-e3a0-4582-aa6c-6251c3b41d00 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:03.785733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1de5a6-4788-4c7e-a245-74f9b8a86d55 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:03.820083 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e5361-61e4-402b-b206-3851b5b0fe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:03.820725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e5361-61e4-402b-b206-3851b5b0fe93 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:03.821036 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e5361-61e4-402b-b206-3851b5b0fe93 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:03.821309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e5361-61e4-402b-b206-3851b5b0fe93 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:03.823342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705923d4-9151-4532-a608-df0be6b6d360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:03.824027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705923d4-9151-4532-a608-df0be6b6d360 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:03.824595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705923d4-9151-4532-a608-df0be6b6d360 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:03.824962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705923d4-9151-4532-a608-df0be6b6d360 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:03.829868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e5361-61e4-402b-b206-3851b5b0fe93 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:03.831370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e5361-61e4-402b-b206-3851b5b0fe93 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-f25e5361-61e4-402b-b206-3851b5b0fe93 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:03.844881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:03.845996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:03.847086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:03.847086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:03.847086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:03.847086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:03.847086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:03.847086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:03.847086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:03.847086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:03.847086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 16:19:03.847086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:03.850328 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 265/527] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:03.850328 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e5361-61e4-402b-b206-3851b5b0fe93 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:03.925142 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705923d4-9151-4532-a608-df0be6b6d360 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:03.926630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705923d4-9151-4532-a608-df0be6b6d360 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:03.977012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e5361-61e4-402b-b206-3851b5b0fe93 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:04.008092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:04.020499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cba4eb-04b2-4389-890d-77379144c771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:04.022962 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cba4eb-04b2-4389-890d-77379144c771 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:04.023479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cba4eb-04b2-4389-890d-77379144c771 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:04.023970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cba4eb-04b2-4389-890d-77379144c771 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:04.025130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cba4eb-04b2-4389-890d-77379144c771 None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=748ad823b944441d907994cda6d8ac7f) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:04.025777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cba4eb-04b2-4389-890d-77379144c771 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:04.027586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cba4eb-04b2-4389-890d-77379144c771 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:04.032028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:04.041083 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 266/528] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:03 2022] GET /identity/v3/role_assignments?user.id=748ad823b944441d907994cda6d8ac7f&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:04.042389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cba4eb-04b2-4389-890d-77379144c771 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:04.047264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 263/529] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:19:04.049232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705923d4-9151-4532-a608-df0be6b6d360 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:04.204875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705923d4-9151-4532-a608-df0be6b6d360 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:04.261783 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cba4eb-04b2-4389-890d-77379144c771 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:04.272230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069d6c1-2e1b-4b1d-b394-30e4948f4db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:04.273319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069d6c1-2e1b-4b1d-b394-30e4948f4db5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:04.273714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069d6c1-2e1b-4b1d-b394-30e4948f4db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:04.274061 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069d6c1-2e1b-4b1d-b394-30e4948f4db5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:04.286519 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069d6c1-2e1b-4b1d-b394-30e4948f4db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:04.288145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069d6c1-2e1b-4b1d-b394-30e4948f4db5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:04.304073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74a14-38be-41dd-a159-d06e7867b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:04.304989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74a14-38be-41dd-a159-d06e7867b174 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:04.305483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74a14-38be-41dd-a159-d06e7867b174 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:04.305983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74a14-38be-41dd-a159-d06e7867b174 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:04.306915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f74a14-38be-41dd-a159-d06e7867b174 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:04.307421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f74a14-38be-41dd-a159-d06e7867b174 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:04.309093 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f74a14-38be-41dd-a159-d06e7867b174 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-6069d6c1-2e1b-4b1d-b394-30e4948f4db5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:04.310309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:04.311184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:04.312023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:04.312023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:04.312023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:04.312023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 264/530] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:04.317395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069d6c1-2e1b-4b1d-b394-30e4948f4db5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:04.332874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 267/531] 10.222.0.55 () {56 vars in 1139 bytes} [Fri Aug 19 16:19:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:04.339957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74a14-38be-41dd-a159-d06e7867b174 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:04.410564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:04.417278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:04.508261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74a14-38be-41dd-a159-d06e7867b174 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:04.519977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069d6c1-2e1b-4b1d-b394-30e4948f4db5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:04.546724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e2b51-d72c-4ba2-a049-b589c3bd94ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:04.547257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e2b51-d72c-4ba2-a049-b589c3bd94ad None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:04.547539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e2b51-d72c-4ba2-a049-b589c3bd94ad None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:04.547808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e2b51-d72c-4ba2-a049-b589c3bd94ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:04.569802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e701b8b-afab-46c7-9e49-0fdd93262aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:04.570876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e701b8b-afab-46c7-9e49-0fdd93262aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:04.571423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e701b8b-afab-46c7-9e49-0fdd93262aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:04.571904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e701b8b-afab-46c7-9e49-0fdd93262aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:04.573331 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e701b8b-afab-46c7-9e49-0fdd93262aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:04.574078 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e701b8b-afab-46c7-9e49-0fdd93262aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:04.576308 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e701b8b-afab-46c7-9e49-0fdd93262aa4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:04.592063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 265/532] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:04.596773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49261c9-b2d5-4381-836c-2040999c3f2d None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:04.596773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49261c9-b2d5-4381-836c-2040999c3f2d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:04.596773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49261c9-b2d5-4381-836c-2040999c3f2d None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:04.596773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 266/533] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:04.597955 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49261c9-b2d5-4381-836c-2040999c3f2d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:04.628062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e2b51-d72c-4ba2-a049-b589c3bd94ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:04.630301 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e2b51-d72c-4ba2-a049-b589c3bd94ad None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:04.700788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:04.725390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49261c9-b2d5-4381-836c-2040999c3f2d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:04.729059 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:04.775352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5b1db-ce5f-4d54-8667-e130006997e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:04.776314 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5b1db-ce5f-4d54-8667-e130006997e9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:04.776808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5b1db-ce5f-4d54-8667-e130006997e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:04.777255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5b1db-ce5f-4d54-8667-e130006997e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:04.787585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 268/534] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 16:19:04.787585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d717ae2-671f-419a-a278-3fd6b352b45a None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:04.787585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d717ae2-671f-419a-a278-3fd6b352b45a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:04.787585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d717ae2-671f-419a-a278-3fd6b352b45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:04.789716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5b1db-ce5f-4d54-8667-e130006997e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:04.791368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5b1db-ce5f-4d54-8667-e130006997e9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-73a5b1db-ce5f-4d54-8667-e130006997e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:04.805147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:04.805987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:04.806837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:04.806837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:04.806837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:04.806837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:04.806837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:04.806837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:04.806837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:04.806837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:04.807231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-2d717ae2-671f-419a-a278-3fd6b352b45a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:04.807231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2d717ae2-671f-419a-a278-3fd6b352b45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:04.812987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 267/535] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:04.814048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5b1db-ce5f-4d54-8667-e130006997e9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:04.900936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 269/536] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:04.902419 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d717ae2-671f-419a-a278-3fd6b352b45a None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:04.934847 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5b1db-ce5f-4d54-8667-e130006997e9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:04.975801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882118fb-07cf-4a97-8eb4-803662a5d55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:04.976936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882118fb-07cf-4a97-8eb4-803662a5d55d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:04.977306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882118fb-07cf-4a97-8eb4-803662a5d55d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:04.977657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882118fb-07cf-4a97-8eb4-803662a5d55d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:04.989549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882118fb-07cf-4a97-8eb4-803662a5d55d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:04.992194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882118fb-07cf-4a97-8eb4-803662a5d55d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-882118fb-07cf-4a97-8eb4-803662a5d55d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:05.010255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:05.011116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:05.011931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:05.011931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:05.011931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:05.011931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 268/537] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:05.017521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882118fb-07cf-4a97-8eb4-803662a5d55d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:05.033668 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:05.037524 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:05.084230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d717ae2-671f-419a-a278-3fd6b352b45a None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:05.142385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882118fb-07cf-4a97-8eb4-803662a5d55d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:05.170037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b59e63-cee8-4968-a8d6-8fa8fdc36779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:05.170750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b59e63-cee8-4968-a8d6-8fa8fdc36779 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:05.171096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b59e63-cee8-4968-a8d6-8fa8fdc36779 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:05.171482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b59e63-cee8-4968-a8d6-8fa8fdc36779 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:05.172250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b59e63-cee8-4968-a8d6-8fa8fdc36779 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:05.172619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b59e63-cee8-4968-a8d6-8fa8fdc36779 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:05.174423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b59e63-cee8-4968-a8d6-8fa8fdc36779 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:05.183670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321bc77-935c-40c5-938a-cfd8274433dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:05.184538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321bc77-935c-40c5-938a-cfd8274433dc None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:05.185023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321bc77-935c-40c5-938a-cfd8274433dc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:05.185495 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321bc77-935c-40c5-938a-cfd8274433dc None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:05.186342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321bc77-935c-40c5-938a-cfd8274433dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:05.186825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321bc77-935c-40c5-938a-cfd8274433dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:05.188982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321bc77-935c-40c5-938a-cfd8274433dc None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:05.199206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 270/538] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:05.203624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 269/539] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:05.208083 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b59e63-cee8-4968-a8d6-8fa8fdc36779 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:05.209233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321bc77-935c-40c5-938a-cfd8274433dc None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:05.296974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:05.327502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321bc77-935c-40c5-938a-cfd8274433dc None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:05.367906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8327705-0245-41f7-b5c2-4ced1ad842d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:05.368977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8327705-0245-41f7-b5c2-4ced1ad842d4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:05.369466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8327705-0245-41f7-b5c2-4ced1ad842d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:05.369989 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8327705-0245-41f7-b5c2-4ced1ad842d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:05.377793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8327705-0245-41f7-b5c2-4ced1ad842d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:05.379673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8327705-0245-41f7-b5c2-4ced1ad842d4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:05.386506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b59e63-cee8-4968-a8d6-8fa8fdc36779 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-f8327705-0245-41f7-b5c2-4ced1ad842d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:05.391928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:05.392962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:05.393991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:05.393991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:05.393991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:05.393991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:05.393991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:05.393991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:05.393991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:05.393991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:05.395289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 270/540] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:05.401114 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8327705-0245-41f7-b5c2-4ced1ad842d4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:05.431604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a520aa4-b873-41b7-8699-cb3014092a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:05.432992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a520aa4-b873-41b7-8699-cb3014092a43 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:05.433625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a520aa4-b873-41b7-8699-cb3014092a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:05.434066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a520aa4-b873-41b7-8699-cb3014092a43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:05.435612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a520aa4-b873-41b7-8699-cb3014092a43 None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=1eaa3980ad8840879e4e426b00e3c56f) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:05.436095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a520aa4-b873-41b7-8699-cb3014092a43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:05.438034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a520aa4-b873-41b7-8699-cb3014092a43 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:05.442343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:05.446747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 271/541] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:05 2022] GET /identity/v3/role_assignments?user.id=1eaa3980ad8840879e4e426b00e3c56f&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:05.447820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a520aa4-b873-41b7-8699-cb3014092a43 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:05.538419 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8327705-0245-41f7-b5c2-4ced1ad842d4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:05.611074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153561a7-4c1a-4567-b93b-5b8096f3250a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:05.611727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153561a7-4c1a-4567-b93b-5b8096f3250a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:05.612013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153561a7-4c1a-4567-b93b-5b8096f3250a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:05.612285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153561a7-4c1a-4567-b93b-5b8096f3250a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:05.617055 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:05.628570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153561a7-4c1a-4567-b93b-5b8096f3250a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-153561a7-4c1a-4567-b93b-5b8096f3250a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:05.634748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:05.635623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:05.636974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:05.636974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:05.636974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:05.636974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:05.636974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:05.636974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:05.636974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:05.636974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:05.636974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:05.636974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:05.636974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:05.636974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:05.637830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a520aa4-b873-41b7-8699-cb3014092a43 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:05.638230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 271/542] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:05.653863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153561a7-4c1a-4567-b93b-5b8096f3250a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:05.673276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa127b3-420e-45cb-9972-874b9c5bac2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:05.673884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa127b3-420e-45cb-9972-874b9c5bac2b None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:05.674159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa127b3-420e-45cb-9972-874b9c5bac2b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:05.674531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa127b3-420e-45cb-9972-874b9c5bac2b None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:05.675222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa127b3-420e-45cb-9972-874b9c5bac2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:05.675517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa127b3-420e-45cb-9972-874b9c5bac2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:05.677083 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa127b3-420e-45cb-9972-874b9c5bac2b None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:05.690692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 272/543] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:05.702534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa127b3-420e-45cb-9972-874b9c5bac2b None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:05.814850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:05.840666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153561a7-4c1a-4567-b93b-5b8096f3250a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:05.865854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa127b3-420e-45cb-9972-874b9c5bac2b None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:05.887203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef909ca-b0bf-4d68-8a7d-3b8bf236d610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:05.887732 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef909ca-b0bf-4d68-8a7d-3b8bf236d610 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:05.888008 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef909ca-b0bf-4d68-8a7d-3b8bf236d610 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:05.888278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef909ca-b0bf-4d68-8a7d-3b8bf236d610 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:05.889403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef909ca-b0bf-4d68-8a7d-3b8bf236d610 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:05.889707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef909ca-b0bf-4d68-8a7d-3b8bf236d610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:05.891588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef909ca-b0bf-4d68-8a7d-3b8bf236d610 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:05.903093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 272/544] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:05.904280 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84914fee-9e3b-4368-a968-26d506c6e747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:05.904500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c616f-afb7-427e-9efb-8317527a11cb None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:05.904671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c616f-afb7-427e-9efb-8317527a11cb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:05.904900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c616f-afb7-427e-9efb-8317527a11cb None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:05.905055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84914fee-9e3b-4368-a968-26d506c6e747 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:05.905216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84914fee-9e3b-4368-a968-26d506c6e747 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:05.905444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84914fee-9e3b-4368-a968-26d506c6e747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:05.906171 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84914fee-9e3b-4368-a968-26d506c6e747 None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=9e0b7efc488a42ffab8ba7e2d101ced4) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:05.906397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 273/545] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:05.906534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84914fee-9e3b-4368-a968-26d506c6e747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:05.907926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84914fee-9e3b-4368-a968-26d506c6e747 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:05.914119 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe185c-b48e-451b-827b-90844b1d9ea0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:05.914404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe185c-b48e-451b-827b-90844b1d9ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:05.914681 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe185c-b48e-451b-827b-90844b1d9ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:05.921167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 273/546] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:05 2022] GET /identity/v3/role_assignments?user.id=9e0b7efc488a42ffab8ba7e2d101ced4&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:05.922220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84914fee-9e3b-4368-a968-26d506c6e747 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:05.942455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-92fe185c-b48e-451b-827b-90844b1d9ea0 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:05.943803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-92fe185c-b48e-451b-827b-90844b1d9ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:06.045430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:06.081969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 274/547] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:06.104254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84914fee-9e3b-4368-a968-26d506c6e747 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:06.128181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:06.141001 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a83aa-eb9a-460a-8296-e3cc5a3d13de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:06.142043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a83aa-eb9a-460a-8296-e3cc5a3d13de None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:06.142521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a83aa-eb9a-460a-8296-e3cc5a3d13de None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:06.143037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a83aa-eb9a-460a-8296-e3cc5a3d13de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:06.155161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a83aa-eb9a-460a-8296-e3cc5a3d13de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:06.156983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a83aa-eb9a-460a-8296-e3cc5a3d13de None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-ee4a83aa-eb9a-460a-8296-e3cc5a3d13de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:06.171959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:06.172925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:06.173740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:06.173740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:06.173740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:06.175761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 274/548] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:06.181717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe185c-b48e-451b-827b-90844b1d9ea0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:06.306812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe185c-b48e-451b-827b-90844b1d9ea0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:06.309766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff42ec1-f430-47b7-9a50-39522cf6d42a None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:06.310273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff42ec1-f430-47b7-9a50-39522cf6d42a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:06.310739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff42ec1-f430-47b7-9a50-39522cf6d42a None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:06.312304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 275/549] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:06.317125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36208e8-104d-45e8-8735-c05588efecda None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:06.317409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36208e8-104d-45e8-8735-c05588efecda None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:06.317680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36208e8-104d-45e8-8735-c05588efecda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:06.348979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14483e-ef60-44f1-a44f-6b857470545c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:06.350083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14483e-ef60-44f1-a44f-6b857470545c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:06.350589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14483e-ef60-44f1-a44f-6b857470545c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:06.351058 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14483e-ef60-44f1-a44f-6b857470545c None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:06.351907 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14483e-ef60-44f1-a44f-6b857470545c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:06.352404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14483e-ef60-44f1-a44f-6b857470545c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:06.352586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b36208e8-104d-45e8-8735-c05588efecda None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:06.354068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b36208e8-104d-45e8-8735-c05588efecda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:06.354615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14483e-ef60-44f1-a44f-6b857470545c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:06.372148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 275/550] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:06.378976 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14483e-ef60-44f1-a44f-6b857470545c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:06.447062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:06.459200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 276/551] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:06.460173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36208e8-104d-45e8-8735-c05588efecda None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:06.507054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14483e-ef60-44f1-a44f-6b857470545c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:06.542111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12bb08-948e-4d6c-b52f-4564ee7bee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:06.542754 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12bb08-948e-4d6c-b52f-4564ee7bee3b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:06.543032 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12bb08-948e-4d6c-b52f-4564ee7bee3b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:06.543303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12bb08-948e-4d6c-b52f-4564ee7bee3b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:06.551280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12bb08-948e-4d6c-b52f-4564ee7bee3b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:06.553307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12bb08-948e-4d6c-b52f-4564ee7bee3b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-1c12bb08-948e-4d6c-b52f-4564ee7bee3b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:06.564936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:06.566291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:06.567335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:06.567335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:06.567335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:06.567335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:06.567335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:06.567335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:06.567335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:06.567335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:06.567335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 16:19:06.567335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:06.567335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 276/552] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:19:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:06.578190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12bb08-948e-4d6c-b52f-4564ee7bee3b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:06.623619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36208e8-104d-45e8-8735-c05588efecda None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:06.632271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:06.660936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d9b50-a69a-498a-9ae5-54ea7e17039e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:06.662425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d9b50-a69a-498a-9ae5-54ea7e17039e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:06.662920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d9b50-a69a-498a-9ae5-54ea7e17039e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:06.663390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d9b50-a69a-498a-9ae5-54ea7e17039e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:06.674183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d9b50-a69a-498a-9ae5-54ea7e17039e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-8c4d9b50-a69a-498a-9ae5-54ea7e17039e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:06.679769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:06.680661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:06.681662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:06.681662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:06.681662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:06.681662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:06.681662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:06.681662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:06.681662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:06.681662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:06.681662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:06.681662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:06.681662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:06.681662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:06.684238 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 277/553] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:06.689884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d9b50-a69a-498a-9ae5-54ea7e17039e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:06.786177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12bb08-948e-4d6c-b52f-4564ee7bee3b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:06.800585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:06.835775 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9f789-bdbe-43fa-a9a0-3d6694ae081b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:06.837155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9f789-bdbe-43fa-a9a0-3d6694ae081b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:06.837648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9f789-bdbe-43fa-a9a0-3d6694ae081b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:06.838101 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9f789-bdbe-43fa-a9a0-3d6694ae081b None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:06.839272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9f789-bdbe-43fa-a9a0-3d6694ae081b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:06.839922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9f789-bdbe-43fa-a9a0-3d6694ae081b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:06.842292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9f789-bdbe-43fa-a9a0-3d6694ae081b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:06.850559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d9b50-a69a-498a-9ae5-54ea7e17039e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:06.868178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 277/554] 10.222.0.55 () {56 vars in 1135 bytes} [Fri Aug 19 16:19:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:06.873758 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9f789-bdbe-43fa-a9a0-3d6694ae081b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:06.888187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77013c85-6932-44e9-a6f0-ef4ca034e932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:06.888809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77013c85-6932-44e9-a6f0-ef4ca034e932 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:06.889089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77013c85-6932-44e9-a6f0-ef4ca034e932 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:06.889353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77013c85-6932-44e9-a6f0-ef4ca034e932 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:06.890008 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77013c85-6932-44e9-a6f0-ef4ca034e932 None admin] RBAC: Adding query filter params (name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:06.890347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77013c85-6932-44e9-a6f0-ef4ca034e932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:06.891881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77013c85-6932-44e9-a6f0-ef4ca034e932 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:06.904573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 278/555] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:06.905402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77013c85-6932-44e9-a6f0-ef4ca034e932 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:06.981145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:06.986530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9f789-bdbe-43fa-a9a0-3d6694ae081b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:07.023298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1176b-e0a4-41a5-82fd-575f4cc266b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:07.023929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1176b-e0a4-41a5-82fd-575f4cc266b4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.024313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1176b-e0a4-41a5-82fd-575f4cc266b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.024620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1176b-e0a4-41a5-82fd-575f4cc266b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.032241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77013c85-6932-44e9-a6f0-ef4ca034e932 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:07.038367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1176b-e0a4-41a5-82fd-575f4cc266b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-0ba1176b-e0a4-41a5-82fd-575f4cc266b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:07.043791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:07.044670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:07.045501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 278/556] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:07.049476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1176b-e0a4-41a5-82fd-575f4cc266b4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:07.077546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:07.079576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd2c03-02f0-424d-93a2-84f65c41ab38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:07.080295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd2c03-02f0-424d-93a2-84f65c41ab38 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.080620 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd2c03-02f0-424d-93a2-84f65c41ab38 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.080939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd2c03-02f0-424d-93a2-84f65c41ab38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.155854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1176b-e0a4-41a5-82fd-575f4cc266b4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:07.156641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd2c03-02f0-424d-93a2-84f65c41ab38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:07.158225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd2c03-02f0-424d-93a2-84f65c41ab38 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:07.190679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:07.193865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd2ffd-6be7-48f0-ba22-e986a7af9942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:07.195294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd2ffd-6be7-48f0-ba22-e986a7af9942 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.195801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd2ffd-6be7-48f0-ba22-e986a7af9942 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.196274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd2ffd-6be7-48f0-ba22-e986a7af9942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.207236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd2ffd-6be7-48f0-ba22-e986a7af9942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:07.209531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd2ffd-6be7-48f0-ba22-e986a7af9942 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c2bd2ffd-6be7-48f0-ba22-e986a7af9942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:07.225450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:07.226928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:07.228028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:07.228028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:07.228028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:07.230140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 279/557] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:07.231172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd2ffd-6be7-48f0-ba22-e986a7af9942 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:07.281101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 279/558] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:19:07.281101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd2c03-02f0-424d-93a2-84f65c41ab38 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:07.317593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:07.343391 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd2ffd-6be7-48f0-ba22-e986a7af9942 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:07.378941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8398a-de1e-41b0-9ae5-16b21809b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:07.380017 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8398a-de1e-41b0-9ae5-16b21809b7de None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.380615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8398a-de1e-41b0-9ae5-16b21809b7de None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.381203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8398a-de1e-41b0-9ae5-16b21809b7de None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.382146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8398a-de1e-41b0-9ae5-16b21809b7de None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:07.382643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8398a-de1e-41b0-9ae5-16b21809b7de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:07.384334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8398a-de1e-41b0-9ae5-16b21809b7de None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:07.388716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd2c03-02f0-424d-93a2-84f65c41ab38 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:07.393518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 280/559] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:07.394586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8398a-de1e-41b0-9ae5-16b21809b7de None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:07.422791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:07.431335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab9709-04c4-45c2-828a-59bbc857977e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:07.431937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab9709-04c4-45c2-828a-59bbc857977e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.432212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab9709-04c4-45c2-828a-59bbc857977e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.432481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab9709-04c4-45c2-828a-59bbc857977e None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.433168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab9709-04c4-45c2-828a-59bbc857977e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:07.433460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab9709-04c4-45c2-828a-59bbc857977e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:07.434840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab9709-04c4-45c2-828a-59bbc857977e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:07.450889 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 280/560] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:07.451745 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab9709-04c4-45c2-828a-59bbc857977e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:07.511277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8398a-de1e-41b0-9ae5-16b21809b7de None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:07.549247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ba0e2-a769-4df8-b831-8c6e6976b565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:07.550336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ba0e2-a769-4df8-b831-8c6e6976b565 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.550847 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ba0e2-a769-4df8-b831-8c6e6976b565 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.551373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ba0e2-a769-4df8-b831-8c6e6976b565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.556888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab9709-04c4-45c2-828a-59bbc857977e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:07.559850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:07.564734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ba0e2-a769-4df8-b831-8c6e6976b565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:07.566672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ba0e2-a769-4df8-b831-8c6e6976b565 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-d87ba0e2-a769-4df8-b831-8c6e6976b565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:07.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:07.584941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:07.585799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:07.585799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:07.585799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:07.587680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 281/561] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:07.588674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ba0e2-a769-4df8-b831-8c6e6976b565 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:07.602668 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33274211-8cbc-4c99-b02c-380795bcee06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:07.603565 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33274211-8cbc-4c99-b02c-380795bcee06 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.603850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33274211-8cbc-4c99-b02c-380795bcee06 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.604125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33274211-8cbc-4c99-b02c-380795bcee06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.616280 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33274211-8cbc-4c99-b02c-380795bcee06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:07.617794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33274211-8cbc-4c99-b02c-380795bcee06 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-33274211-8cbc-4c99-b02c-380795bcee06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:07.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:07.634324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:07.635170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:07.635170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:07.635170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:07.635170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 281/562] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:07.635170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33274211-8cbc-4c99-b02c-380795bcee06 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:07.687787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:07.705763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ba0e2-a769-4df8-b831-8c6e6976b565 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:07.743305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df264f0-f2e5-4828-8475-e44a921ffa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:07.743927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df264f0-f2e5-4828-8475-e44a921ffa9d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.744212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df264f0-f2e5-4828-8475-e44a921ffa9d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.744483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df264f0-f2e5-4828-8475-e44a921ffa9d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.748800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33274211-8cbc-4c99-b02c-380795bcee06 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:07.752531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df264f0-f2e5-4828-8475-e44a921ffa9d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:07.754026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df264f0-f2e5-4828-8475-e44a921ffa9d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4df264f0-f2e5-4828-8475-e44a921ffa9d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:07.766616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:07.767581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:07.768487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:07.768487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:07.768487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:07.768487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:07.768487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:07.768487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:07.768487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:07.768487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:07.768487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:19:07.768487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:07.768487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 282/563] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Aug 19 16:19:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:07.769130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a9048-1b54-4d8b-8a20-ff13fc8c525e None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.769265 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a9048-1b54-4d8b-8a20-ff13fc8c525e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.769535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a9048-1b54-4d8b-8a20-ff13fc8c525e None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.771643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 283/564] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:07.772353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a9048-1b54-4d8b-8a20-ff13fc8c525e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:07.790711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab2630-8d3b-4188-9a17-a8055da9dcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:07.791615 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab2630-8d3b-4188-9a17-a8055da9dcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.792078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab2630-8d3b-4188-9a17-a8055da9dcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.792533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab2630-8d3b-4188-9a17-a8055da9dcb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.793400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab2630-8d3b-4188-9a17-a8055da9dcb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:07.793903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab2630-8d3b-4188-9a17-a8055da9dcb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:07.795443 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab2630-8d3b-4188-9a17-a8055da9dcb9 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:07.812684 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 282/565] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:07.813778 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab2630-8d3b-4188-9a17-a8055da9dcb9 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:07.826429 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:07.884665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a9048-1b54-4d8b-8a20-ff13fc8c525e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:07.929801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe24ce-01d4-400d-ad8f-cec6300af78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:07.930828 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe24ce-01d4-400d-ad8f-cec6300af78a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.931311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe24ce-01d4-400d-ad8f-cec6300af78a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.931768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe24ce-01d4-400d-ad8f-cec6300af78a None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.932616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe24ce-01d4-400d-ad8f-cec6300af78a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:07.933144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe24ce-01d4-400d-ad8f-cec6300af78a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:07.934595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe24ce-01d4-400d-ad8f-cec6300af78a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:07.935936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:07.939287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab2630-8d3b-4188-9a17-a8055da9dcb9 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:07.949559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 284/566] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:07.951222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b388cdc-931c-4193-91f1-22b2f4f77bd4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.951569 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b388cdc-931c-4193-91f1-22b2f4f77bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.951913 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b388cdc-931c-4193-91f1-22b2f4f77bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.971010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3b388cdc-931c-4193-91f1-22b2f4f77bd4 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:07.972309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3b388cdc-931c-4193-91f1-22b2f4f77bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:07.980942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf428dde-2240-4e1d-b5c6-be7633e30b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:07.981753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf428dde-2240-4e1d-b5c6-be7633e30b72 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:07.982232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf428dde-2240-4e1d-b5c6-be7633e30b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:07.982649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf428dde-2240-4e1d-b5c6-be7633e30b72 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:07.983660 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf428dde-2240-4e1d-b5c6-be7633e30b72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:07.984111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf428dde-2240-4e1d-b5c6-be7633e30b72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:07.985820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf428dde-2240-4e1d-b5c6-be7633e30b72 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:08.005226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 283/567] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Aug 19 16:19:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:08.006861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf428dde-2240-4e1d-b5c6-be7633e30b72 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:08.052776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 285/568] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:08.052776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b388cdc-931c-4193-91f1-22b2f4f77bd4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:08.096290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:08.129226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf428dde-2240-4e1d-b5c6-be7633e30b72 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:08.166340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086c88a-e388-47be-91ec-61d127a81d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:08.166888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086c88a-e388-47be-91ec-61d127a81d53 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:08.167168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086c88a-e388-47be-91ec-61d127a81d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:08.167440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086c88a-e388-47be-91ec-61d127a81d53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:08.168487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b388cdc-931c-4193-91f1-22b2f4f77bd4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:08.174309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086c88a-e388-47be-91ec-61d127a81d53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:08.175756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086c88a-e388-47be-91ec-61d127a81d53 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-b086c88a-e388-47be-91ec-61d127a81d53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:08.187618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:08.188453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:08.190010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:08.190010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:08.190010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:08.190010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:08.190010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:08.190010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:08.190010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:08.190010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:08.190010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 284/569] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:08.190010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086c88a-e388-47be-91ec-61d127a81d53 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:08.211588 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:08.211981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d291ac-95c7-4162-a2a6-6b4e56532097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:08.212617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d291ac-95c7-4162-a2a6-6b4e56532097 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:08.212932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d291ac-95c7-4162-a2a6-6b4e56532097 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:08.213208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d291ac-95c7-4162-a2a6-6b4e56532097 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:08.220804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d291ac-95c7-4162-a2a6-6b4e56532097 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:08.222245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d291ac-95c7-4162-a2a6-6b4e56532097 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a3d291ac-95c7-4162-a2a6-6b4e56532097 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:08.233961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:08.234930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:08.235853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:08.235853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:08.235853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:08.235853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:08.235853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:08.235853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:08.235853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:08.235853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:08.235853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 19 16:19:08.235853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:08.239671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 286/570] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:08.241602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95637740-ce34-4916-ad83-65f209ed65de None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:08.241899 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95637740-ce34-4916-ad83-65f209ed65de None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:08.242179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95637740-ce34-4916-ad83-65f209ed65de None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:08.243433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 287/571] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:08.244123 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95637740-ce34-4916-ad83-65f209ed65de None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:08.306893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086c88a-e388-47be-91ec-61d127a81d53 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:08.342470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb445198-af03-4fae-be07-6f977b2eeea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:08.343196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb445198-af03-4fae-be07-6f977b2eeea1 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:08.343479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb445198-af03-4fae-be07-6f977b2eeea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:08.343737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb445198-af03-4fae-be07-6f977b2eeea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:08.346158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:08.351059 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95637740-ce34-4916-ad83-65f209ed65de None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:08.356438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb445198-af03-4fae-be07-6f977b2eeea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:08.357902 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb445198-af03-4fae-be07-6f977b2eeea1 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-eb445198-af03-4fae-be07-6f977b2eeea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:08.372663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:08.373591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:08.374459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:08.374459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:08.374459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:08.374459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 285/572] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:08.374675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb445198-af03-4fae-be07-6f977b2eeea1 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:08.390387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb5b7f-0776-45d6-a581-1c94ec066e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:08.391004 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb5b7f-0776-45d6-a581-1c94ec066e15 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:08.391285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb5b7f-0776-45d6-a581-1c94ec066e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:08.391559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb5b7f-0776-45d6-a581-1c94ec066e15 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:08.392257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb5b7f-0776-45d6-a581-1c94ec066e15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:08.392559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb5b7f-0776-45d6-a581-1c94ec066e15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:08.394043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb5b7f-0776-45d6-a581-1c94ec066e15 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:08.406369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 288/573] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:08.407598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd16669-267f-472b-a3c1-c5502d05de8f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:08.407883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd16669-267f-472b-a3c1-c5502d05de8f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:08.408155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd16669-267f-472b-a3c1-c5502d05de8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:08.425954 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ccd16669-267f-472b-a3c1-c5502d05de8f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:08.427081 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ccd16669-267f-472b-a3c1-c5502d05de8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:08.446818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:08.488953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb445198-af03-4fae-be07-6f977b2eeea1 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:08.501067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 289/574] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:08.501934 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd16669-267f-472b-a3c1-c5502d05de8f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:08.550510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a1289-a2af-40e9-b82c-7e757590ece7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:08.551444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a1289-a2af-40e9-b82c-7e757590ece7 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:08.551833 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a1289-a2af-40e9-b82c-7e757590ece7 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:08.552111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a1289-a2af-40e9-b82c-7e757590ece7 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:08.553532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a1289-a2af-40e9-b82c-7e757590ece7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:08.554025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a1289-a2af-40e9-b82c-7e757590ece7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:08.556114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a1289-a2af-40e9-b82c-7e757590ece7 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:08.582117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 286/575] 10.222.0.55 () {56 vars in 1137 bytes} [Fri Aug 19 16:19:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:08.582117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a1289-a2af-40e9-b82c-7e757590ece7 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:08.629714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:08.639613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd16669-267f-472b-a3c1-c5502d05de8f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:08.686438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3da3de-3bbe-44a0-b0db-01532f694614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:08.687514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3da3de-3bbe-44a0-b0db-01532f694614 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:08.687992 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3da3de-3bbe-44a0-b0db-01532f694614 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:08.688467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3da3de-3bbe-44a0-b0db-01532f694614 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:08.703532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3da3de-3bbe-44a0-b0db-01532f694614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-ce3da3de-3bbe-44a0-b0db-01532f694614 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:08.708273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:08.709731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:08.710535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 290/576] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:08.711229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3da3de-3bbe-44a0-b0db-01532f694614 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:08.719323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a1289-a2af-40e9-b82c-7e757590ece7 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:08.759533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:08.771738 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ef21-551d-451e-b5ee-afb19e9544d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:08.772176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ef21-551d-451e-b5ee-afb19e9544d7 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:08.772521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ef21-551d-451e-b5ee-afb19e9544d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:08.772728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ef21-551d-451e-b5ee-afb19e9544d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:08.773388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428ef21-551d-451e-b5ee-afb19e9544d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:08.773683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428ef21-551d-451e-b5ee-afb19e9544d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:08.775273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428ef21-551d-451e-b5ee-afb19e9544d7 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:08.791163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 287/577] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:08.791994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ef21-551d-451e-b5ee-afb19e9544d7 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:08.834398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3da3de-3bbe-44a0-b0db-01532f694614 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:08.873569 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a129697-75d6-4f75-acaa-b56e83a98afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:08.874173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a129697-75d6-4f75-acaa-b56e83a98afe None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:08.874451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a129697-75d6-4f75-acaa-b56e83a98afe None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:08.874723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a129697-75d6-4f75-acaa-b56e83a98afe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:08.875441 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a129697-75d6-4f75-acaa-b56e83a98afe None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=c755f9538e984bc3877112896595e158) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:08.875742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a129697-75d6-4f75-acaa-b56e83a98afe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:08.877252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a129697-75d6-4f75-acaa-b56e83a98afe None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:08.882536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 291/578] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:08 2022] GET /identity/v3/role_assignments?user.id=c755f9538e984bc3877112896595e158&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:08.883302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a129697-75d6-4f75-acaa-b56e83a98afe None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:08.894118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ef21-551d-451e-b5ee-afb19e9544d7 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:08.911420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:08.933538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e1348-7166-4ba5-8118-b363352c1fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:08.934190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e1348-7166-4ba5-8118-b363352c1fdb None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:08.934469 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e1348-7166-4ba5-8118-b363352c1fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:08.934741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e1348-7166-4ba5-8118-b363352c1fdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:08.947735 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e1348-7166-4ba5-8118-b363352c1fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:08.949308 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e1348-7166-4ba5-8118-b363352c1fdb None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-f52e1348-7166-4ba5-8118-b363352c1fdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:08.966418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:08.967606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:08.968456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:08.968456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:08.968456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:08.968456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 288/579] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:08.968836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e1348-7166-4ba5-8118-b363352c1fdb None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:09.012278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a129697-75d6-4f75-acaa-b56e83a98afe None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:09.016155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:09.049868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9824162-647b-451a-99d7-de2b36ac9246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:09.050728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9824162-647b-451a-99d7-de2b36ac9246 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:09.051176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9824162-647b-451a-99d7-de2b36ac9246 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:09.051603 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9824162-647b-451a-99d7-de2b36ac9246 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:09.052325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9824162-647b-451a-99d7-de2b36ac9246 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:09.052630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9824162-647b-451a-99d7-de2b36ac9246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:09.054023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9824162-647b-451a-99d7-de2b36ac9246 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:09.067297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 292/580] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:09.068844 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9824162-647b-451a-99d7-de2b36ac9246 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:09.090025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e1348-7166-4ba5-8118-b363352c1fdb None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:09.129520 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7d9a0-d767-4810-abec-3f507d20b38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:09.130154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7d9a0-d767-4810-abec-3f507d20b38e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:09.130950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7d9a0-d767-4810-abec-3f507d20b38e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:09.131435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7d9a0-d767-4810-abec-3f507d20b38e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:09.142149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7d9a0-d767-4810-abec-3f507d20b38e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-8bc7d9a0-d767-4810-abec-3f507d20b38e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:09.148322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:09.149291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:09.150119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:09.150119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:09.150119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:09.150119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:09.150119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:09.150119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:09.150119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:09.150119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:09.150119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:09.150119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:09.150119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:09.150119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:09.153156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 289/581] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:09.154175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:09.155401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7d9a0-d767-4810-abec-3f507d20b38e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:09.179650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9824162-647b-451a-99d7-de2b36ac9246 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:09.217514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722917fb-8322-4ef6-9285-607db6e59fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:09.218074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722917fb-8322-4ef6-9285-607db6e59fba None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:09.218350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722917fb-8322-4ef6-9285-607db6e59fba None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:09.218613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722917fb-8322-4ef6-9285-607db6e59fba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:09.225901 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722917fb-8322-4ef6-9285-607db6e59fba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:09.227360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722917fb-8322-4ef6-9285-607db6e59fba None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-722917fb-8322-4ef6-9285-607db6e59fba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:09.238611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:09.239471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:09.240469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:09.240469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:09.240469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:09.240469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:09.240469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:09.240469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:09.240469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:09.240469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:09.244387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 293/582] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:09.245272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722917fb-8322-4ef6-9285-607db6e59fba None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:09.269305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:09.273287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7d9a0-d767-4810-abec-3f507d20b38e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:09.310608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935756e6-c454-4210-8b68-232045176808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:09.311214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935756e6-c454-4210-8b68-232045176808 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:09.311493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935756e6-c454-4210-8b68-232045176808 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:09.311765 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935756e6-c454-4210-8b68-232045176808 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:09.312402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935756e6-c454-4210-8b68-232045176808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:09.312877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935756e6-c454-4210-8b68-232045176808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:09.314718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935756e6-c454-4210-8b68-232045176808 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:09.326407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 290/583] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:09.327232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935756e6-c454-4210-8b68-232045176808 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:09.357763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722917fb-8322-4ef6-9285-607db6e59fba None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:09.395468 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91d04e-bc75-4b2b-b7f1-1967e847537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:09.397435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91d04e-bc75-4b2b-b7f1-1967e847537c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:09.397724 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91d04e-bc75-4b2b-b7f1-1967e847537c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:09.397999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91d04e-bc75-4b2b-b7f1-1967e847537c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:09.402385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:09.412410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91d04e-bc75-4b2b-b7f1-1967e847537c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:09.414272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91d04e-bc75-4b2b-b7f1-1967e847537c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:09.429227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935756e6-c454-4210-8b68-232045176808 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-df91d04e-bc75-4b2b-b7f1-1967e847537c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:09.430404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:09.431283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:09.432139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:09.432139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:09.432139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:09.432139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 294/584] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:09.432410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91d04e-bc75-4b2b-b7f1-1967e847537c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:09.469830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8b18a-a954-438a-af19-745dd3582b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:09.470353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8b18a-a954-438a-af19-745dd3582b96 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:09.470793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8b18a-a954-438a-af19-745dd3582b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:09.471063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8b18a-a954-438a-af19-745dd3582b96 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:09.471717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8b18a-a954-438a-af19-745dd3582b96 None admin] RBAC: Adding query filter params (name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:09.472014 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8b18a-a954-438a-af19-745dd3582b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:09.473395 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8b18a-a954-438a-af19-745dd3582b96 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:09.482792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 291/585] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:09.483849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8b18a-a954-438a-af19-745dd3582b96 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:09.511343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:09.545913 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91d04e-bc75-4b2b-b7f1-1967e847537c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:09.581018 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b0375-4a71-4d43-b7c6-3e2e56e13725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:09.581616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b0375-4a71-4d43-b7c6-3e2e56e13725 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:09.581891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b0375-4a71-4d43-b7c6-3e2e56e13725 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:09.582162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b0375-4a71-4d43-b7c6-3e2e56e13725 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:09.582841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b0375-4a71-4d43-b7c6-3e2e56e13725 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:09.583139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b0375-4a71-4d43-b7c6-3e2e56e13725 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:09.584591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b0375-4a71-4d43-b7c6-3e2e56e13725 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:09.590820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8b18a-a954-438a-af19-745dd3582b96 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:09.601424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 295/586] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:09.601424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b0375-4a71-4d43-b7c6-3e2e56e13725 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:09.631793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75165321-39a9-4423-aa56-016c7300f165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:09.632416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75165321-39a9-4423-aa56-016c7300f165 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:09.632706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75165321-39a9-4423-aa56-016c7300f165 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:09.633002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75165321-39a9-4423-aa56-016c7300f165 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:09.641785 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75165321-39a9-4423-aa56-016c7300f165 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:09.644101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75165321-39a9-4423-aa56-016c7300f165 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-75165321-39a9-4423-aa56-016c7300f165 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:09.656293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:09.657462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:09.658324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:09.658324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:09.658324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:09.658324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:09.658324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:09.658324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:09.658324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:09.658324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:09.660093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:09.662053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 292/587] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:09.662879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75165321-39a9-4423-aa56-016c7300f165 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:09.713990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b0375-4a71-4d43-b7c6-3e2e56e13725 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:09.749631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:09.750133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e8cfe-fd62-4523-8ee2-58b9d803a603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:09.750856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e8cfe-fd62-4523-8ee2-58b9d803a603 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:09.751185 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e8cfe-fd62-4523-8ee2-58b9d803a603 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:09.751616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e8cfe-fd62-4523-8ee2-58b9d803a603 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:09.752366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e8cfe-fd62-4523-8ee2-58b9d803a603 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:09.752936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e8cfe-fd62-4523-8ee2-58b9d803a603 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:09.754598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e8cfe-fd62-4523-8ee2-58b9d803a603 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:09.766831 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 296/588] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:09.767733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e8cfe-fd62-4523-8ee2-58b9d803a603 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:09.775628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75165321-39a9-4423-aa56-016c7300f165 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:09.815787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271ed1c-9b89-4583-b392-3cc8c2af821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:09.816418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271ed1c-9b89-4583-b392-3cc8c2af821a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:09.816695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271ed1c-9b89-4583-b392-3cc8c2af821a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:09.816994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271ed1c-9b89-4583-b392-3cc8c2af821a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:09.828514 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271ed1c-9b89-4583-b392-3cc8c2af821a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:09.830019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271ed1c-9b89-4583-b392-3cc8c2af821a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-b271ed1c-9b89-4583-b392-3cc8c2af821a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:09.843998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:09.845143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:09.846031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:09.846031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:09.846031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:09.846031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 293/589] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:09.846362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271ed1c-9b89-4583-b392-3cc8c2af821a None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:09.880737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e8cfe-fd62-4523-8ee2-58b9d803a603 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:09.890572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:09.918272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03762849-b4fb-4883-8a29-21177bdf5d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:09.918928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03762849-b4fb-4883-8a29-21177bdf5d67 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:09.919208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03762849-b4fb-4883-8a29-21177bdf5d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:09.919499 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03762849-b4fb-4883-8a29-21177bdf5d67 None admin] PATH_INFO: `/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/4739e80619a54b178b9aacec6a42a7f0/roles/3a9f17a4e0074293989007ce1a8b2565` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:09.936393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03762849-b4fb-4883-8a29-21177bdf5d67 None admin] RBAC: Authorizing `identity:create_grant(project_id=d143cf444f354ec0bf2089dae3d94255, user_id=4739e80619a54b178b9aacec6a42a7f0, role_id=3a9f17a4e0074293989007ce1a8b2565)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:09.938028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03762849-b4fb-4883-8a29-21177bdf5d67 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:09.951625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271ed1c-9b89-4583-b392-3cc8c2af821a None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:09.961037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 297/590] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:19:09 2022] PUT /identity/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/4739e80619a54b178b9aacec6a42a7f0/roles/3a9f17a4e0074293989007ce1a8b2565 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:19:09.962138 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03762849-b4fb-4883-8a29-21177bdf5d67 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:09.994005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd36a85-1883-45d1-ac29-814803f58942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:09.994788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd36a85-1883-45d1-ac29-814803f58942 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:09.995346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd36a85-1883-45d1-ac29-814803f58942 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:09.995714 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd36a85-1883-45d1-ac29-814803f58942 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:09.996931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd36a85-1883-45d1-ac29-814803f58942 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:09.997058 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd36a85-1883-45d1-ac29-814803f58942 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:09.998637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd36a85-1883-45d1-ac29-814803f58942 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:10.000982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:10.019778 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 294/591] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:10.019778 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd36a85-1883-45d1-ac29-814803f58942 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:10.096212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03762849-b4fb-4883-8a29-21177bdf5d67 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:10.133464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd36a85-1883-45d1-ac29-814803f58942 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:10.138026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3fa7d-0c2f-4090-95e9-ac2ec396c872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:10.138748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3fa7d-0c2f-4090-95e9-ac2ec396c872 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:10.139105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3fa7d-0c2f-4090-95e9-ac2ec396c872 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:10.139453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3fa7d-0c2f-4090-95e9-ac2ec396c872 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:10.147204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3fa7d-0c2f-4090-95e9-ac2ec396c872 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:10.148940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3fa7d-0c2f-4090-95e9-ac2ec396c872 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-43c3fa7d-0c2f-4090-95e9-ac2ec396c872 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:10.160684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:10.162194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:10.163274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:10.163274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:10.163274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:10.163274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:10.163274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:10.163274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:10.163274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:10.163274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:10.163274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 16:19:10.163274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:10.164659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 298/592] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:10.168804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3fa7d-0c2f-4090-95e9-ac2ec396c872 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:10.184715 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0f6f7-ae44-4782-85cd-724ac10615da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:10.185606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f6f7-ae44-4782-85cd-724ac10615da None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:10.186082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f6f7-ae44-4782-85cd-724ac10615da None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:10.186504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f6f7-ae44-4782-85cd-724ac10615da None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:10.187326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0f6f7-ae44-4782-85cd-724ac10615da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:10.187838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:10.188509 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0f6f7-ae44-4782-85cd-724ac10615da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:10.190139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0f6f7-ae44-4782-85cd-724ac10615da None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:10.205557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 295/593] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:10.206592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0f6f7-ae44-4782-85cd-724ac10615da None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:10.271609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:10.290317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3fa7d-0c2f-4090-95e9-ac2ec396c872 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:10.323912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0f6f7-ae44-4782-85cd-724ac10615da None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:10.327355 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c05d1a-f8fd-4476-bf65-026f2a0f6854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:10.327530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05d1a-f8fd-4476-bf65-026f2a0f6854 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:10.327928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05d1a-f8fd-4476-bf65-026f2a0f6854 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:10.328308 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05d1a-f8fd-4476-bf65-026f2a0f6854 None admin] PATH_INFO: `/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/748ad823b944441d907994cda6d8ac7f/roles/3a9f17a4e0074293989007ce1a8b2565` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:10.344217 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c05d1a-f8fd-4476-bf65-026f2a0f6854 None admin] RBAC: Authorizing `identity:create_grant(project_id=d143cf444f354ec0bf2089dae3d94255, user_id=748ad823b944441d907994cda6d8ac7f, role_id=3a9f17a4e0074293989007ce1a8b2565)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:10.345889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c05d1a-f8fd-4476-bf65-026f2a0f6854 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:10.362819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 299/594] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:19:10 2022] PUT /identity/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/748ad823b944441d907994cda6d8ac7f/roles/3a9f17a4e0074293989007ce1a8b2565 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:19:10.370166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80855064-d519-429c-aaa3-d269a6242800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:10.370992 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c05d1a-f8fd-4476-bf65-026f2a0f6854 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:10.372267 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80855064-d519-429c-aaa3-d269a6242800 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:10.373133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80855064-d519-429c-aaa3-d269a6242800 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:10.374172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80855064-d519-429c-aaa3-d269a6242800 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:10.376135 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80855064-d519-429c-aaa3-d269a6242800 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:10.376671 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80855064-d519-429c-aaa3-d269a6242800 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:10.379512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80855064-d519-429c-aaa3-d269a6242800 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:10.424413 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 296/595] 10.222.0.55 () {56 vars in 1137 bytes} [Fri Aug 19 16:19:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:10.431014 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80855064-d519-429c-aaa3-d269a6242800 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:10.486455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:10.564078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80855064-d519-429c-aaa3-d269a6242800 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:10.567676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c05d1a-f8fd-4476-bf65-026f2a0f6854 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:10.580004 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:10.605911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6e18a-bfe1-41e7-9af7-9fa10978589f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:10.606804 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6e18a-bfe1-41e7-9af7-9fa10978589f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:10.607313 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6e18a-bfe1-41e7-9af7-9fa10978589f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:10.607859 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6e18a-bfe1-41e7-9af7-9fa10978589f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:10.622702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e147068-8b17-4c5c-83d9-1e8e91a2e786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:10.623346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e147068-8b17-4c5c-83d9-1e8e91a2e786 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:10.623629 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e147068-8b17-4c5c-83d9-1e8e91a2e786 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:10.623904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e147068-8b17-4c5c-83d9-1e8e91a2e786 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:10.627354 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6e18a-bfe1-41e7-9af7-9fa10978589f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:10.629311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6e18a-bfe1-41e7-9af7-9fa10978589f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:10.633512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e147068-8b17-4c5c-83d9-1e8e91a2e786 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:10.634978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e147068-8b17-4c5c-83d9-1e8e91a2e786 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-f4e6e18a-bfe1-41e7-9af7-9fa10978589f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:10.646189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:10.647188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:10.648033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:10.648033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:10.648033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-2e147068-8b17-4c5c-83d9-1e8e91a2e786 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:10.648219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:10.649065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:10.649978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:10.649978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:10.649978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:10.649978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:10.649978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:10.649978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:10.649978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:10.649978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:10.649978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 16:19:10.649978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:10.649978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 300/596] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:10.653517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 297/597] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:10.658753 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e147068-8b17-4c5c-83d9-1e8e91a2e786 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:10.659838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6e18a-bfe1-41e7-9af7-9fa10978589f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:10.770517 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e147068-8b17-4c5c-83d9-1e8e91a2e786 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:10.773438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6e18a-bfe1-41e7-9af7-9fa10978589f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:10.799091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:10.808044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81a40f-776c-4cd2-9bfb-dfcccb935ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:10.808684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81a40f-776c-4cd2-9bfb-dfcccb935ddb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:10.808981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81a40f-776c-4cd2-9bfb-dfcccb935ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:10.809260 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81a40f-776c-4cd2-9bfb-dfcccb935ddb None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:10.809908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81a40f-776c-4cd2-9bfb-dfcccb935ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:10.810230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81a40f-776c-4cd2-9bfb-dfcccb935ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:10.811608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81a40f-776c-4cd2-9bfb-dfcccb935ddb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:10.816070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145e89f-4968-48f6-b55e-2c40c34cfb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:10.817090 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145e89f-4968-48f6-b55e-2c40c34cfb9d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:10.817578 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145e89f-4968-48f6-b55e-2c40c34cfb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:10.818088 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145e89f-4968-48f6-b55e-2c40c34cfb9d None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:10.819023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145e89f-4968-48f6-b55e-2c40c34cfb9d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:10.819518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145e89f-4968-48f6-b55e-2c40c34cfb9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:10.821180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145e89f-4968-48f6-b55e-2c40c34cfb9d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:10.826029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 301/598] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:10.827296 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cf511-3403-4de4-a3d4-672da4653acf None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:10.827585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cf511-3403-4de4-a3d4-672da4653acf None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:10.827863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cf511-3403-4de4-a3d4-672da4653acf None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:10.833730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 302/599] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:10.834536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cf511-3403-4de4-a3d4-672da4653acf None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:10.840118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 298/600] 10.222.0.55 () {56 vars in 1139 bytes} [Fri Aug 19 16:19:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:10.841405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7d93d-da67-4149-93d7-b942bd2855fd None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:10.841696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7d93d-da67-4149-93d7-b942bd2855fd None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:10.841969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7d93d-da67-4149-93d7-b942bd2855fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:10.863223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-bda7d93d-da67-4149-93d7-b942bd2855fd None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:10.864508 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-bda7d93d-da67-4149-93d7-b942bd2855fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:10.886750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:10.935437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 299/601] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:10.936420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7d93d-da67-4149-93d7-b942bd2855fd None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:10.951810 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cf511-3403-4de4-a3d4-672da4653acf None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:10.990757 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432133c-aa89-41b4-a2c8-5400a615452a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:10.991669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432133c-aa89-41b4-a2c8-5400a615452a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:10.992115 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432133c-aa89-41b4-a2c8-5400a615452a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:10.992568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432133c-aa89-41b4-a2c8-5400a615452a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:11.000301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432133c-aa89-41b4-a2c8-5400a615452a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:11.001951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432133c-aa89-41b4-a2c8-5400a615452a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-9432133c-aa89-41b4-a2c8-5400a615452a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:11.014483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:11.015389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:11.016268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:11.016268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:11.016268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:11.016268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:11.016268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:11.016268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:11.016268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:11.016268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:11.018864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 303/602] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:11.025610 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432133c-aa89-41b4-a2c8-5400a615452a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:11.040417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7d93d-da67-4149-93d7-b942bd2855fd None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:11.079064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b6e4e-e05e-4fe0-97ae-25c07bc670ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:11.079064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b6e4e-e05e-4fe0-97ae-25c07bc670ab None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:11.079064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b6e4e-e05e-4fe0-97ae-25c07bc670ab None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:11.079064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b6e4e-e05e-4fe0-97ae-25c07bc670ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:11.090378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b6e4e-e05e-4fe0-97ae-25c07bc670ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:11.091958 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b6e4e-e05e-4fe0-97ae-25c07bc670ab None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-b59b6e4e-e05e-4fe0-97ae-25c07bc670ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:11.107609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:11.108637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:11.109562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:11.109562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:11.109562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:11.109562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 300/603] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:11.117927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b6e4e-e05e-4fe0-97ae-25c07bc670ab None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:11.120039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:11.125903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:11.205804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432133c-aa89-41b4-a2c8-5400a615452a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:11.255869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b6e4e-e05e-4fe0-97ae-25c07bc670ab None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:11.263088 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f35fff-591d-4f7a-b97e-54946b0f2edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:11.263828 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f35fff-591d-4f7a-b97e-54946b0f2edb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:11.264112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f35fff-591d-4f7a-b97e-54946b0f2edb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:11.264389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f35fff-591d-4f7a-b97e-54946b0f2edb None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:11.265139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f35fff-591d-4f7a-b97e-54946b0f2edb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:11.265444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f35fff-591d-4f7a-b97e-54946b0f2edb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:11.267068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f35fff-591d-4f7a-b97e-54946b0f2edb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:11.281456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 304/604] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:11.282416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f35fff-591d-4f7a-b97e-54946b0f2edb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:11.293273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9b6f2-bd9d-4015-b970-48853256c313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:11.293875 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9b6f2-bd9d-4015-b970-48853256c313 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:11.294150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9b6f2-bd9d-4015-b970-48853256c313 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:11.294420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9b6f2-bd9d-4015-b970-48853256c313 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:11.295054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9b6f2-bd9d-4015-b970-48853256c313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:11.295351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9b6f2-bd9d-4015-b970-48853256c313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:11.296712 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9b6f2-bd9d-4015-b970-48853256c313 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:11.309440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 301/605] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:11.310241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9b6f2-bd9d-4015-b970-48853256c313 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:11.384927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:11.393181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f35fff-591d-4f7a-b97e-54946b0f2edb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:11.406431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:11.426715 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9b6f2-bd9d-4015-b970-48853256c313 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:11.439099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410702-7d8a-4d23-a205-23319f35d765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:11.440893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410702-7d8a-4d23-a205-23319f35d765 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:11.441348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410702-7d8a-4d23-a205-23319f35d765 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:11.442162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410702-7d8a-4d23-a205-23319f35d765 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:11.456157 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d410702-7d8a-4d23-a205-23319f35d765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-1d410702-7d8a-4d23-a205-23319f35d765 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:11.464665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:11.465787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:11.466794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:11.466794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:11.466794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:11.466794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:11.466794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:11.466794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:11.466794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:11.466794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:11.466794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:11.466794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:11.466794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:11.466794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:11.474113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 305/606] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:11.474113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410702-7d8a-4d23-a205-23319f35d765 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:11.476482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60ef0-0e48-4d06-8281-aef7f7c6f442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:11.477297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60ef0-0e48-4d06-8281-aef7f7c6f442 None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:11.477604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60ef0-0e48-4d06-8281-aef7f7c6f442 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:11.478170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60ef0-0e48-4d06-8281-aef7f7c6f442 None admin] PATH_INFO: `/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/1eaa3980ad8840879e4e426b00e3c56f/roles/3a9f17a4e0074293989007ce1a8b2565` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:11.496338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b60ef0-0e48-4d06-8281-aef7f7c6f442 None admin] RBAC: Authorizing `identity:create_grant(project_id=d143cf444f354ec0bf2089dae3d94255, user_id=1eaa3980ad8840879e4e426b00e3c56f, role_id=3a9f17a4e0074293989007ce1a8b2565)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:11.498385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b60ef0-0e48-4d06-8281-aef7f7c6f442 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:11.518439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 302/607] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:19:11 2022] PUT /identity/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/1eaa3980ad8840879e4e426b00e3c56f/roles/3a9f17a4e0074293989007ce1a8b2565 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:19:11.519340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60ef0-0e48-4d06-8281-aef7f7c6f442 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:11.591748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410702-7d8a-4d23-a205-23319f35d765 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:11.631971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad3eb6-26db-4127-ab3e-1d2056344c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:11.633080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad3eb6-26db-4127-ab3e-1d2056344c78 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:11.633593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad3eb6-26db-4127-ab3e-1d2056344c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:11.634302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad3eb6-26db-4127-ab3e-1d2056344c78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:11.636948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60ef0-0e48-4d06-8281-aef7f7c6f442 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:11.641967 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad3eb6-26db-4127-ab3e-1d2056344c78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:11.643756 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad3eb6-26db-4127-ab3e-1d2056344c78 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:11.656951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-0dad3eb6-26db-4127-ab3e-1d2056344c78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:11.659657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:11.660503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:11.661660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:11.661660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:11.661660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:11.661660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:11.661660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:11.661660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:11.661660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:11.661660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:11.668535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 306/608] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:11.670647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad3eb6-26db-4127-ab3e-1d2056344c78 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:11.686266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe53285-fe7e-4efc-b5eb-cced1646b67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:11.687644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe53285-fe7e-4efc-b5eb-cced1646b67a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:11.688185 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe53285-fe7e-4efc-b5eb-cced1646b67a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:11.688647 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe53285-fe7e-4efc-b5eb-cced1646b67a None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:11.689686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe53285-fe7e-4efc-b5eb-cced1646b67a None admin] RBAC: Adding query filter params (name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:11.690207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe53285-fe7e-4efc-b5eb-cced1646b67a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:11.692415 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe53285-fe7e-4efc-b5eb-cced1646b67a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:11.704291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 303/609] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:11.710141 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:11.717568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe53285-fe7e-4efc-b5eb-cced1646b67a None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:11.817368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad3eb6-26db-4127-ab3e-1d2056344c78 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:11.859745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844908b2-945b-4398-b809-b77b0d77723c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:11.860605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844908b2-945b-4398-b809-b77b0d77723c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:11.860933 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844908b2-945b-4398-b809-b77b0d77723c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:11.861210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844908b2-945b-4398-b809-b77b0d77723c None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:11.861943 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844908b2-945b-4398-b809-b77b0d77723c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:11.862538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844908b2-945b-4398-b809-b77b0d77723c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:11.865248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844908b2-945b-4398-b809-b77b0d77723c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:11.867204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe53285-fe7e-4efc-b5eb-cced1646b67a None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:11.884778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 307/610] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:11.886886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844908b2-945b-4398-b809-b77b0d77723c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:11.907843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af67d47-6c6e-4c59-8757-8d21bb73a4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:11.908407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af67d47-6c6e-4c59-8757-8d21bb73a4ea None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:11.908688 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af67d47-6c6e-4c59-8757-8d21bb73a4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:11.908992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af67d47-6c6e-4c59-8757-8d21bb73a4ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:11.920810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af67d47-6c6e-4c59-8757-8d21bb73a4ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:11.922210 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af67d47-6c6e-4c59-8757-8d21bb73a4ea None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-4af67d47-6c6e-4c59-8757-8d21bb73a4ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:11.938187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:11.939141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:11.940405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:11.940405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:11.940405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:11.940405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 304/611] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:11.946058 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af67d47-6c6e-4c59-8757-8d21bb73a4ea None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:11.976437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:12.017174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:12.023220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844908b2-945b-4398-b809-b77b0d77723c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:12.107023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af67d47-6c6e-4c59-8757-8d21bb73a4ea None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:12.109485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec122569-9e39-4aa5-b179-59275f3fe878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:12.110175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec122569-9e39-4aa5-b179-59275f3fe878 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:12.110458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec122569-9e39-4aa5-b179-59275f3fe878 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:12.110730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec122569-9e39-4aa5-b179-59275f3fe878 None admin] PATH_INFO: `/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/9e0b7efc488a42ffab8ba7e2d101ced4/roles/3a9f17a4e0074293989007ce1a8b2565` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:12.143069 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f56b2-7193-4998-a4f6-4975a25817d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:12.143685 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f56b2-7193-4998-a4f6-4975a25817d4 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:12.143969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f56b2-7193-4998-a4f6-4975a25817d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:12.144241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f56b2-7193-4998-a4f6-4975a25817d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:12.144909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f56b2-7193-4998-a4f6-4975a25817d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:12.145346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f56b2-7193-4998-a4f6-4975a25817d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:12.146828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f56b2-7193-4998-a4f6-4975a25817d4 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:12.159621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 305/612] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:12.165365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f56b2-7193-4998-a4f6-4975a25817d4 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:12.166103 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec122569-9e39-4aa5-b179-59275f3fe878 None admin] RBAC: Authorizing `identity:create_grant(project_id=d143cf444f354ec0bf2089dae3d94255, user_id=9e0b7efc488a42ffab8ba7e2d101ced4, role_id=3a9f17a4e0074293989007ce1a8b2565)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:12.167571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec122569-9e39-4aa5-b179-59275f3fe878 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:12.207593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 308/613] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:19:11 2022] PUT /identity/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/9e0b7efc488a42ffab8ba7e2d101ced4/roles/3a9f17a4e0074293989007ce1a8b2565 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:19:12.291546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:12.306351 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b28909-3e4b-4a65-88e7-816acee6ec00 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:12.306640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b28909-3e4b-4a65-88e7-816acee6ec00 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:12.306912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b28909-3e4b-4a65-88e7-816acee6ec00 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:12.308412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 309/614] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:12.309756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f56b2-7193-4998-a4f6-4975a25817d4 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:12.318339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2da299-cfda-4189-ad7b-3a68aaa56f97 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:12.318675 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2da299-cfda-4189-ad7b-3a68aaa56f97 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:12.318886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2da299-cfda-4189-ad7b-3a68aaa56f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:12.347683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee90cf-5887-411d-904e-cc2e210c4d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:12.348311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee90cf-5887-411d-904e-cc2e210c4d8b None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:12.348592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee90cf-5887-411d-904e-cc2e210c4d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:12.349414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee90cf-5887-411d-904e-cc2e210c4d8b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:12.358667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee90cf-5887-411d-904e-cc2e210c4d8b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:12.360158 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee90cf-5887-411d-904e-cc2e210c4d8b None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:12.366563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3b2da299-cfda-4189-ad7b-3a68aaa56f97 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:12.367800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3b2da299-cfda-4189-ad7b-3a68aaa56f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-ccee90cf-5887-411d-904e-cc2e210c4d8b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:12.374359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:12.375341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:12.376254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:12.376254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:12.376254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:12.376254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:12.376254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:12.376254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:12.376254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:12.376254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:12.376254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 16:19:12.376254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:12.376254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 306/615] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:19:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:12.381586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee90cf-5887-411d-904e-cc2e210c4d8b None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:12.469113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 310/616] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:12.514693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee90cf-5887-411d-904e-cc2e210c4d8b None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:12.551951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1079ffc-74cb-4f46-830c-5292d0128c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:12.552672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1079ffc-74cb-4f46-830c-5292d0128c0a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:12.552981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1079ffc-74cb-4f46-830c-5292d0128c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:12.553351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1079ffc-74cb-4f46-830c-5292d0128c0a None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:12.554124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1079ffc-74cb-4f46-830c-5292d0128c0a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:12.554433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1079ffc-74cb-4f46-830c-5292d0128c0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:12.555882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1079ffc-74cb-4f46-830c-5292d0128c0a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:12.574601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 307/617] 10.222.0.55 () {56 vars in 1135 bytes} [Fri Aug 19 16:19:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:12.580255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2da299-cfda-4189-ad7b-3a68aaa56f97 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:12.661184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:12.692941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2da299-cfda-4189-ad7b-3a68aaa56f97 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:12.730585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6156e8-cecc-40f2-aea6-c52177ab7997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:12.731235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6156e8-cecc-40f2-aea6-c52177ab7997 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:12.731588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6156e8-cecc-40f2-aea6-c52177ab7997 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:12.731930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6156e8-cecc-40f2-aea6-c52177ab7997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:12.743991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6156e8-cecc-40f2-aea6-c52177ab7997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:12.745644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6156e8-cecc-40f2-aea6-c52177ab7997 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-0c6156e8-cecc-40f2-aea6-c52177ab7997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:12.761174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:12.762408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:12.763476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:12.763476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:12.763476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:12.768843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 311/618] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:12.769901 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1079ffc-74cb-4f46-830c-5292d0128c0a None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:12.780100 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:12.880675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1079ffc-74cb-4f46-830c-5292d0128c0a None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:12.918244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa9a2c-bde6-44f1-b4fd-ae467fa0c5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:12.918956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa9a2c-bde6-44f1-b4fd-ae467fa0c5dd None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:12.919309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa9a2c-bde6-44f1-b4fd-ae467fa0c5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:12.919658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa9a2c-bde6-44f1-b4fd-ae467fa0c5dd None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:12.920445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa9a2c-bde6-44f1-b4fd-ae467fa0c5dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:12.920855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa9a2c-bde6-44f1-b4fd-ae467fa0c5dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:12.922300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa9a2c-bde6-44f1-b4fd-ae467fa0c5dd None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:12.934833 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6156e8-cecc-40f2-aea6-c52177ab7997 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:12.936992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 308/619] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:12.954232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa9a2c-bde6-44f1-b4fd-ae467fa0c5dd None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:13.048122 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:13.061243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6156e8-cecc-40f2-aea6-c52177ab7997 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:13.083918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:13.098349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa9a2c-bde6-44f1-b4fd-ae467fa0c5dd None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:13.100812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a293390-674b-48fb-b1e1-df5955e1f3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:13.101617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a293390-674b-48fb-b1e1-df5955e1f3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.102069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a293390-674b-48fb-b1e1-df5955e1f3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.102523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a293390-674b-48fb-b1e1-df5955e1f3f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.113484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a293390-674b-48fb-b1e1-df5955e1f3f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-3a293390-674b-48fb-b1e1-df5955e1f3f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:13.119902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:13.121166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:13.122084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:13.122084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:13.122084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:13.122084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:13.122084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:13.122084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:13.122084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:13.122084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:13.122084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:13.122084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:13.122084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:13.122084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:13.124366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 312/620] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:13.125738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bf9fe-b188-45dc-b93d-fd1e3cc7100b None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.126168 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bf9fe-b188-45dc-b93d-fd1e3cc7100b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.126579 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bf9fe-b188-45dc-b93d-fd1e3cc7100b None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.128207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 313/621] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:13.134007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bf9fe-b188-45dc-b93d-fd1e3cc7100b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:13.141427 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4280316-39a3-4084-8ae9-f351fcc239e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:13.142066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4280316-39a3-4084-8ae9-f351fcc239e2 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.142349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4280316-39a3-4084-8ae9-f351fcc239e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.142624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4280316-39a3-4084-8ae9-f351fcc239e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.151659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4280316-39a3-4084-8ae9-f351fcc239e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:13.153899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4280316-39a3-4084-8ae9-f351fcc239e2 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-f4280316-39a3-4084-8ae9-f351fcc239e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:13.169472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:13.170547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:13.171596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:13.171596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:13.171596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:13.171596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:13.171596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:13.171596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:13.171596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:13.171596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:13.172826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 309/622] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:13.174226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e8e25-bed6-4859-87b7-ffe71cb909dd None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.174638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e8e25-bed6-4859-87b7-ffe71cb909dd None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.175038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e8e25-bed6-4859-87b7-ffe71cb909dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.193787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-826e8e25-bed6-4859-87b7-ffe71cb909dd None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:13.195184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-826e8e25-bed6-4859-87b7-ffe71cb909dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:13.253310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bf9fe-b188-45dc-b93d-fd1e3cc7100b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:13.270531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 310/623] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:13.272144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e8e25-bed6-4859-87b7-ffe71cb909dd None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:13.290000 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b6915-5eca-4626-bfd2-ce247bae6e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:13.290618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b6915-5eca-4626-bfd2-ce247bae6e22 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.290903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b6915-5eca-4626-bfd2-ce247bae6e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.291183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b6915-5eca-4626-bfd2-ce247bae6e22 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.291847 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b6915-5eca-4626-bfd2-ce247bae6e22 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:13.292447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b6915-5eca-4626-bfd2-ce247bae6e22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:13.293559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b6915-5eca-4626-bfd2-ce247bae6e22 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:13.304875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 314/624] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:13.309580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b6915-5eca-4626-bfd2-ce247bae6e22 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:13.384213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:13.386493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:13.443801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b6915-5eca-4626-bfd2-ce247bae6e22 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:13.482093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfbf5d-416f-4b97-b9c3-1faf3037ccd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:13.482746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfbf5d-416f-4b97-b9c3-1faf3037ccd0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.483037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfbf5d-416f-4b97-b9c3-1faf3037ccd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.483320 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfbf5d-416f-4b97-b9c3-1faf3037ccd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.494750 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfbf5d-416f-4b97-b9c3-1faf3037ccd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:13.496918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfbf5d-416f-4b97-b9c3-1faf3037ccd0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:13.507695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e8e25-bed6-4859-87b7-ffe71cb909dd None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c6bfbf5d-416f-4b97-b9c3-1faf3037ccd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:13.514024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:13.515188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:13.516272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:13.516272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:13.516272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:13.524532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 315/625] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:13.530257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfbf5d-416f-4b97-b9c3-1faf3037ccd0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:13.580728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f6f19-d7d5-4068-b905-2307277d4a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:13.581296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f6f19-d7d5-4068-b905-2307277d4a18 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.581571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f6f19-d7d5-4068-b905-2307277d4a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.581843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f6f19-d7d5-4068-b905-2307277d4a18 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.582536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f6f19-d7d5-4068-b905-2307277d4a18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:13.582834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f6f19-d7d5-4068-b905-2307277d4a18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:13.584948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f6f19-d7d5-4068-b905-2307277d4a18 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:13.607884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 311/626] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:13.609119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f6f19-d7d5-4068-b905-2307277d4a18 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:13.664345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:13.672025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfbf5d-416f-4b97-b9c3-1faf3037ccd0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:13.713588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc0a4c-d61f-467d-883f-79e163148982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:13.714209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc0a4c-d61f-467d-883f-79e163148982 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.714490 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc0a4c-d61f-467d-883f-79e163148982 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.714820 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc0a4c-d61f-467d-883f-79e163148982 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.715567 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc0a4c-d61f-467d-883f-79e163148982 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:13.715874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc0a4c-d61f-467d-883f-79e163148982 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:13.717381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc0a4c-d61f-467d-883f-79e163148982 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:13.725436 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f6f19-d7d5-4068-b905-2307277d4a18 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:13.731372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 316/627] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:13.732190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc0a4c-d61f-467d-883f-79e163148982 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:13.742747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:13.761915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2a19a-4d4f-4913-9cc7-1328f456eb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:13.762690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2a19a-4d4f-4913-9cc7-1328f456eb04 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.763029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2a19a-4d4f-4913-9cc7-1328f456eb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.763359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2a19a-4d4f-4913-9cc7-1328f456eb04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.774880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2a19a-4d4f-4913-9cc7-1328f456eb04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-b6b2a19a-4d4f-4913-9cc7-1328f456eb04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:13.780148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:13.782107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 312/628] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:13.783881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2a19a-4d4f-4913-9cc7-1328f456eb04 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:13.849318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc0a4c-d61f-467d-883f-79e163148982 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:13.885509 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2a19a-4d4f-4913-9cc7-1328f456eb04 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:13.887927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f58a71-2ee5-4f23-84b9-56f736ff1452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:13.888651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58a71-2ee5-4f23-84b9-56f736ff1452 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.888961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58a71-2ee5-4f23-84b9-56f736ff1452 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.889233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58a71-2ee5-4f23-84b9-56f736ff1452 None admin] PATH_INFO: `/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/c755f9538e984bc3877112896595e158/roles/3a9f17a4e0074293989007ce1a8b2565` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.905434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f58a71-2ee5-4f23-84b9-56f736ff1452 None admin] RBAC: Authorizing `identity:create_grant(project_id=d143cf444f354ec0bf2089dae3d94255, user_id=c755f9538e984bc3877112896595e158, role_id=3a9f17a4e0074293989007ce1a8b2565)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:13.906997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f58a71-2ee5-4f23-84b9-56f736ff1452 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:13.921448 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5575ea5-9fbb-4339-a5e0-ee1747de5c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:13.922072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5575ea5-9fbb-4339-a5e0-ee1747de5c1d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.922352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5575ea5-9fbb-4339-a5e0-ee1747de5c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.922523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 317/629] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:19:13 2022] PUT /identity/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/c755f9538e984bc3877112896595e158/roles/3a9f17a4e0074293989007ce1a8b2565 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:19:13.922678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5575ea5-9fbb-4339-a5e0-ee1747de5c1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.923352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f58a71-2ee5-4f23-84b9-56f736ff1452 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:13.928640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:13.935228 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5575ea5-9fbb-4339-a5e0-ee1747de5c1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:13.936686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5575ea5-9fbb-4339-a5e0-ee1747de5c1d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-e5575ea5-9fbb-4339-a5e0-ee1747de5c1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:13.951246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:13.952297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:13.953250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:13.953250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:13.953250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:13.953250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:13.953250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:13.953250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:13.953250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:13.953250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:13.953250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:19:13.953250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:13.955318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 313/630] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Aug 19 16:19:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:13.956588 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e8636-79df-4693-a4fa-ea4232bdb475 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.956895 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e8636-79df-4693-a4fa-ea4232bdb475 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.957166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e8636-79df-4693-a4fa-ea4232bdb475 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.958604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 314/631] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:13.968887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7cf1d-9394-4fd4-be35-fbde1b3bb64f None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:13.968887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7cf1d-9394-4fd4-be35-fbde1b3bb64f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:13.968887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7cf1d-9394-4fd4-be35-fbde1b3bb64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:13.985055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-eeb7cf1d-9394-4fd4-be35-fbde1b3bb64f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:13.986938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-eeb7cf1d-9394-4fd4-be35-fbde1b3bb64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:14.030541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:14.059033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f58a71-2ee5-4f23-84b9-56f736ff1452 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:14.093211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 315/632] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:14.094266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7cf1d-9394-4fd4-be35-fbde1b3bb64f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:14.100256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe6325-4290-4b18-adfc-bf250e4ebef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:14.100862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe6325-4290-4b18-adfc-bf250e4ebef7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:14.101120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe6325-4290-4b18-adfc-bf250e4ebef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:14.101395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe6325-4290-4b18-adfc-bf250e4ebef7 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:14.102074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe6325-4290-4b18-adfc-bf250e4ebef7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:14.102377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe6325-4290-4b18-adfc-bf250e4ebef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:14.104146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe6325-4290-4b18-adfc-bf250e4ebef7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:14.114036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 318/633] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:14.122870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe6325-4290-4b18-adfc-bf250e4ebef7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:14.214489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:14.237391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7cf1d-9394-4fd4-be35-fbde1b3bb64f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:14.253067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe6325-4290-4b18-adfc-bf250e4ebef7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:14.281523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da618f-2700-494a-b6df-a649839a61ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:14.282714 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da618f-2700-494a-b6df-a649839a61ff None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:14.283674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da618f-2700-494a-b6df-a649839a61ff None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:14.284163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da618f-2700-494a-b6df-a649839a61ff None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:14.285120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da618f-2700-494a-b6df-a649839a61ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:14.285755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da618f-2700-494a-b6df-a649839a61ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:14.287433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da618f-2700-494a-b6df-a649839a61ff None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:14.308850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 316/634] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Aug 19 16:19:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:14.314487 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da618f-2700-494a-b6df-a649839a61ff None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:14.334418 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b792b0-c276-4812-b123-2367b4a1078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:14.335398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b792b0-c276-4812-b123-2367b4a1078f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:14.335877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b792b0-c276-4812-b123-2367b4a1078f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:14.336387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b792b0-c276-4812-b123-2367b4a1078f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:14.349759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b792b0-c276-4812-b123-2367b4a1078f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:14.351641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b792b0-c276-4812-b123-2367b4a1078f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-12b792b0-c276-4812-b123-2367b4a1078f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:14.370221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:14.371187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:14.372049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:14.372049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:14.372049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:14.374253 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 319/635] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:14.377774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:14.379445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b792b0-c276-4812-b123-2367b4a1078f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:14.466048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da618f-2700-494a-b6df-a649839a61ff None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:14.499363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b792b0-c276-4812-b123-2367b4a1078f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:14.504808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfcecc-0535-4b34-b39c-da0e88f79250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:14.506162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfcecc-0535-4b34-b39c-da0e88f79250 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:14.506792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfcecc-0535-4b34-b39c-da0e88f79250 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:14.507416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfcecc-0535-4b34-b39c-da0e88f79250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:14.519690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:14.542669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cffe8-7066-4ca1-86c7-78dd5bb34f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:14.543526 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cffe8-7066-4ca1-86c7-78dd5bb34f83 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:14.543986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cffe8-7066-4ca1-86c7-78dd5bb34f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:14.544500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cffe8-7066-4ca1-86c7-78dd5bb34f83 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:14.545482 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cffe8-7066-4ca1-86c7-78dd5bb34f83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:14.545920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cffe8-7066-4ca1-86c7-78dd5bb34f83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:14.547464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cffe8-7066-4ca1-86c7-78dd5bb34f83 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:14.561825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 320/636] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:14.562891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cffe8-7066-4ca1-86c7-78dd5bb34f83 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:14.580609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfcecc-0535-4b34-b39c-da0e88f79250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:14.581858 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfcecc-0535-4b34-b39c-da0e88f79250 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:14.608766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:14.673831 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cffe8-7066-4ca1-86c7-78dd5bb34f83 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:14.701992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 317/637] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:19:14.703409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfcecc-0535-4b34-b39c-da0e88f79250 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:14.712052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff603d-dfcd-4960-bb7b-82d4b00c2266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:14.713136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff603d-dfcd-4960-bb7b-82d4b00c2266 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:14.713424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff603d-dfcd-4960-bb7b-82d4b00c2266 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:14.713700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff603d-dfcd-4960-bb7b-82d4b00c2266 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:14.727944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff603d-dfcd-4960-bb7b-82d4b00c2266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4dff603d-dfcd-4960-bb7b-82d4b00c2266 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:14.734427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:14.735428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:14.736232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:14.736232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:14.736232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:14.736232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:14.736232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:14.736232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:14.736232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:14.736232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:14.736232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:14.736232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:14.736232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:14.736232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:14.736925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 321/638] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:14.737613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab19d55-6334-4645-8a22-d5ac733f5ca3 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:14.737896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab19d55-6334-4645-8a22-d5ac733f5ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:14.738225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab19d55-6334-4645-8a22-d5ac733f5ca3 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:14.743679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 322/639] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:14.743679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab19d55-6334-4645-8a22-d5ac733f5ca3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:14.798189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:14.810912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfcecc-0535-4b34-b39c-da0e88f79250 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:14.838239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:14.850624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7c149-5f1e-45b6-8740-51ac2b87319d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:14.851245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7c149-5f1e-45b6-8740-51ac2b87319d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:14.851520 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7c149-5f1e-45b6-8740-51ac2b87319d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:14.851786 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7c149-5f1e-45b6-8740-51ac2b87319d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:14.859451 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7c149-5f1e-45b6-8740-51ac2b87319d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:14.860836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7c149-5f1e-45b6-8740-51ac2b87319d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:14.866605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab19d55-6334-4645-8a22-d5ac733f5ca3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-4df7c149-5f1e-45b6-8740-51ac2b87319d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:14.872095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:14.873034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:14.873969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:14.873969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:14.873969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:14.873969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:14.873969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:14.873969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:14.873969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:14.873969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:14.873969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 19 16:19:14.873969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:14.873969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 318/640] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:14.874870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c32dd-02e9-4b99-976d-cf03656c12f1 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:14.874966 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c32dd-02e9-4b99-976d-cf03656c12f1 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:14.875160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c32dd-02e9-4b99-976d-cf03656c12f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:14.892400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-6c8c32dd-02e9-4b99-976d-cf03656c12f1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:14.893616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6c8c32dd-02e9-4b99-976d-cf03656c12f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:14.909338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d7cbd-8f20-4a86-ac85-298648e1bfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:14.909973 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d7cbd-8f20-4a86-ac85-298648e1bfca None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:14.910259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d7cbd-8f20-4a86-ac85-298648e1bfca None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:14.910535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d7cbd-8f20-4a86-ac85-298648e1bfca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:14.922456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d7cbd-8f20-4a86-ac85-298648e1bfca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:14.923903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d7cbd-8f20-4a86-ac85-298648e1bfca None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-057d7cbd-8f20-4a86-ac85-298648e1bfca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:14.938898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:14.939918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:14.940803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:14.940803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:14.940803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:14.944967 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 323/641] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:14.945645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d7cbd-8f20-4a86-ac85-298648e1bfca None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:14.964587 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 319/642] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:14.965569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8c32dd-02e9-4b99-976d-cf03656c12f1 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:15.071262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:15.074423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8c32dd-02e9-4b99-976d-cf03656c12f1 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:15.084114 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d7cbd-8f20-4a86-ac85-298648e1bfca None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:15.096876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:15.125050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd19e8a-5733-4b65-a969-8faa028b079e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:15.125679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd19e8a-5733-4b65-a969-8faa028b079e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:15.126042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd19e8a-5733-4b65-a969-8faa028b079e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:15.126377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd19e8a-5733-4b65-a969-8faa028b079e None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:15.127184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd19e8a-5733-4b65-a969-8faa028b079e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:15.127559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd19e8a-5733-4b65-a969-8faa028b079e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:15.129102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd19e8a-5733-4b65-a969-8faa028b079e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:15.129454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854647b-ac64-4f42-924c-9b7d1992d072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:15.131214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854647b-ac64-4f42-924c-9b7d1992d072 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:15.131730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854647b-ac64-4f42-924c-9b7d1992d072 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:15.132178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854647b-ac64-4f42-924c-9b7d1992d072 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:15.133170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854647b-ac64-4f42-924c-9b7d1992d072 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:15.133671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854647b-ac64-4f42-924c-9b7d1992d072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:15.135259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854647b-ac64-4f42-924c-9b7d1992d072 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:15.144504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 324/643] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:15.146433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854647b-ac64-4f42-924c-9b7d1992d072 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:15.155821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 320/644] 10.222.0.55 () {56 vars in 1137 bytes} [Fri Aug 19 16:19:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:15.156780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd19e8a-5733-4b65-a969-8faa028b079e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:15.269502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854647b-ac64-4f42-924c-9b7d1992d072 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:15.286688 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd19e8a-5733-4b65-a969-8faa028b079e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:15.306774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690955a-47e3-452f-b6ed-365d32c03e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:15.307379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690955a-47e3-452f-b6ed-365d32c03e5d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:15.307656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690955a-47e3-452f-b6ed-365d32c03e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:15.307926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690955a-47e3-452f-b6ed-365d32c03e5d None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:15.308564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690955a-47e3-452f-b6ed-365d32c03e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:15.308890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690955a-47e3-452f-b6ed-365d32c03e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:15.310274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690955a-47e3-452f-b6ed-365d32c03e5d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:15.322840 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 325/645] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:15.323626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690955a-47e3-452f-b6ed-365d32c03e5d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:15.328795 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a87ae9-c108-44a1-9c17-b5a2adf21b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:15.329434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a87ae9-c108-44a1-9c17-b5a2adf21b33 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:15.329711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a87ae9-c108-44a1-9c17-b5a2adf21b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:15.329982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a87ae9-c108-44a1-9c17-b5a2adf21b33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:15.338995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:15.341138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a87ae9-c108-44a1-9c17-b5a2adf21b33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:15.342904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a87ae9-c108-44a1-9c17-b5a2adf21b33 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-71a87ae9-c108-44a1-9c17-b5a2adf21b33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:15.357927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:15.358905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:15.359861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:15.359861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:15.359861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:15.359861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 321/646] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:15.360092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a87ae9-c108-44a1-9c17-b5a2adf21b33 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:15.386019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:15.433708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690955a-47e3-452f-b6ed-365d32c03e5d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:15.467786 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a87ae9-c108-44a1-9c17-b5a2adf21b33 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:15.469270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781b385-e775-42f6-8027-737d0980f822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:15.469879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781b385-e775-42f6-8027-737d0980f822 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:15.470156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781b385-e775-42f6-8027-737d0980f822 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:15.470422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781b385-e775-42f6-8027-737d0980f822 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:15.506048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d8c5d-7dc0-4b5f-af94-8089282184c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:15.506664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d8c5d-7dc0-4b5f-af94-8089282184c0 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:15.506937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d8c5d-7dc0-4b5f-af94-8089282184c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:15.507203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d8c5d-7dc0-4b5f-af94-8089282184c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:15.517739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d8c5d-7dc0-4b5f-af94-8089282184c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-5e1d8c5d-7dc0-4b5f-af94-8089282184c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:15.523314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:15.524420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:15.525374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 322/647] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:15.526059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d8c5d-7dc0-4b5f-af94-8089282184c0 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:15.548980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781b385-e775-42f6-8027-737d0980f822 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:15.550819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781b385-e775-42f6-8027-737d0980f822 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:15.566710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:15.614250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:15.635499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d8c5d-7dc0-4b5f-af94-8089282184c0 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:15.666049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 326/648] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:19:15.666898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781b385-e775-42f6-8027-737d0980f822 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:15.679721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e5018-b0b0-4d0f-b5fd-bface838e25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:15.680354 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e5018-b0b0-4d0f-b5fd-bface838e25d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:15.680634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e5018-b0b0-4d0f-b5fd-bface838e25d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:15.681015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e5018-b0b0-4d0f-b5fd-bface838e25d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:15.688724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e5018-b0b0-4d0f-b5fd-bface838e25d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:15.690944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e5018-b0b0-4d0f-b5fd-bface838e25d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-5b1e5018-b0b0-4d0f-b5fd-bface838e25d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:15.703330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:15.704215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:15.705178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:15.705178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:15.705178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:15.705178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:15.705178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:15.705178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:15.705178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:15.705178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:15.705178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 323/649] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:15.705178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e5018-b0b0-4d0f-b5fd-bface838e25d None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:15.778424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781b385-e775-42f6-8027-737d0980f822 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:15.796510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:15.812939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e5018-b0b0-4d0f-b5fd-bface838e25d None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:15.822503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4d409-af2d-4b6d-a26e-86c585dbf5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:15.823130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4d409-af2d-4b6d-a26e-86c585dbf5fd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:15.823411 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4d409-af2d-4b6d-a26e-86c585dbf5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:15.823687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4d409-af2d-4b6d-a26e-86c585dbf5fd None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:15.824342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4d409-af2d-4b6d-a26e-86c585dbf5fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:15.824635 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4d409-af2d-4b6d-a26e-86c585dbf5fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:15.826195 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4d409-af2d-4b6d-a26e-86c585dbf5fd None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:15.839383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 327/650] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:15.840196 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4d409-af2d-4b6d-a26e-86c585dbf5fd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:15.855204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199030a-7bc3-4bda-8fe5-a881a660fd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:15.855837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199030a-7bc3-4bda-8fe5-a881a660fd3e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:15.856090 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199030a-7bc3-4bda-8fe5-a881a660fd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:15.856365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199030a-7bc3-4bda-8fe5-a881a660fd3e None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:15.857062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199030a-7bc3-4bda-8fe5-a881a660fd3e None admin] RBAC: Adding query filter params (name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:15.857378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199030a-7bc3-4bda-8fe5-a881a660fd3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:15.858741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199030a-7bc3-4bda-8fe5-a881a660fd3e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:15.868296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 324/651] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:15.869137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199030a-7bc3-4bda-8fe5-a881a660fd3e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:15.878530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:15.951782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4d409-af2d-4b6d-a26e-86c585dbf5fd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:15.980867 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199030a-7bc3-4bda-8fe5-a881a660fd3e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:15.988183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22445358-e42c-4509-aea6-f3963e0d5a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:15.988960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22445358-e42c-4509-aea6-f3963e0d5a35 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:15.989247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22445358-e42c-4509-aea6-f3963e0d5a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:15.990094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22445358-e42c-4509-aea6-f3963e0d5a35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.001221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22445358-e42c-4509-aea6-f3963e0d5a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:16.003106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22445358-e42c-4509-aea6-f3963e0d5a35 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:16.015500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0526e-806e-413d-aa7c-b9ab7a942b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:16.016098 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0526e-806e-413d-aa7c-b9ab7a942b1d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:16.016374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0526e-806e-413d-aa7c-b9ab7a942b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:16.016642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0526e-806e-413d-aa7c-b9ab7a942b1d None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-22445358-e42c-4509-aea6-f3963e0d5a35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:16.017077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:16.018208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:16.019068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:16.019068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:16.019068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:16.019233 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0526e-806e-413d-aa7c-b9ab7a942b1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:16.019233 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0526e-806e-413d-aa7c-b9ab7a942b1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:16.019368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 328/652] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:16.019368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22445358-e42c-4509-aea6-f3963e0d5a35 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:16.019499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0526e-806e-413d-aa7c-b9ab7a942b1d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:16.031172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 325/653] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:16.031966 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0526e-806e-413d-aa7c-b9ab7a942b1d None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:16.053382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:16.096915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:16.129138 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22445358-e42c-4509-aea6-f3963e0d5a35 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:16.132114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0526e-806e-413d-aa7c-b9ab7a942b1d None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:16.162746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f34aa1-7d34-4684-9362-8e5c53f33968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:16.163366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f34aa1-7d34-4684-9362-8e5c53f33968 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:16.163644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f34aa1-7d34-4684-9362-8e5c53f33968 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:16.163913 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f34aa1-7d34-4684-9362-8e5c53f33968 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.165440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dde71-a5b9-492f-b968-3ccda16d32c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:16.166218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dde71-a5b9-492f-b968-3ccda16d32c9 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:16.166966 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dde71-a5b9-492f-b968-3ccda16d32c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:16.167242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dde71-a5b9-492f-b968-3ccda16d32c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.171175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f34aa1-7d34-4684-9362-8e5c53f33968 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:16.172600 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f34aa1-7d34-4684-9362-8e5c53f33968 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:16.177077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dde71-a5b9-492f-b968-3ccda16d32c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:16.178990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dde71-a5b9-492f-b968-3ccda16d32c9 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-66f34aa1-7d34-4684-9362-8e5c53f33968 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:16.184596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:16.185546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:16.186453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:16.186453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:16.186453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:16.186453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:16.186453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:16.186453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:16.186453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:16.186453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:16.186453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 16:19:16.186453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:16.186453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 329/654] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:16.187031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd06b4e-efe5-423f-90e8-df8d63af7d14 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:16.187337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd06b4e-efe5-423f-90e8-df8d63af7d14 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:16.187525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd06b4e-efe5-423f-90e8-df8d63af7d14 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.188738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 330/655] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:16.191558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd06b4e-efe5-423f-90e8-df8d63af7d14 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-1e2dde71-a5b9-492f-b968-3ccda16d32c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:16.198207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:16.199335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:16.200438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:16.200438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:16.200438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:16.200438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 326/656] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:16.200438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dde71-a5b9-492f-b968-3ccda16d32c9 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:16.276456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:16.293975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd06b4e-efe5-423f-90e8-df8d63af7d14 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:16.300587 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dde71-a5b9-492f-b968-3ccda16d32c9 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:16.317811 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:16.327237 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79518a84-0e64-4aa0-bb49-c59d15f252bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:16.327842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79518a84-0e64-4aa0-bb49-c59d15f252bd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:16.328123 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79518a84-0e64-4aa0-bb49-c59d15f252bd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:16.328396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79518a84-0e64-4aa0-bb49-c59d15f252bd None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.329073 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79518a84-0e64-4aa0-bb49-c59d15f252bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:16.329374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79518a84-0e64-4aa0-bb49-c59d15f252bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:16.330759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79518a84-0e64-4aa0-bb49-c59d15f252bd None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:16.335088 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22a041-f971-4bf9-ab99-64afc9a52fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:16.335674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22a041-f971-4bf9-ab99-64afc9a52fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:16.335948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22a041-f971-4bf9-ab99-64afc9a52fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:16.336220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22a041-f971-4bf9-ab99-64afc9a52fc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.336953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22a041-f971-4bf9-ab99-64afc9a52fc6 None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=4739e80619a54b178b9aacec6a42a7f0) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:16.337255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22a041-f971-4bf9-ab99-64afc9a52fc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:16.338657 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22a041-f971-4bf9-ab99-64afc9a52fc6 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:16.342794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 331/657] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:16.343421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79518a84-0e64-4aa0-bb49-c59d15f252bd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:16.344789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 327/658] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:16 2022] GET /identity/v3/role_assignments?user.id=4739e80619a54b178b9aacec6a42a7f0&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:16.345388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d2b98-2a83-4530-b6fc-b4b6849eb604 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:16.345671 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d2b98-2a83-4530-b6fc-b4b6849eb604 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:16.345941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d2b98-2a83-4530-b6fc-b4b6849eb604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.366059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-fa4d2b98-2a83-4530-b6fc-b4b6849eb604 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:16.367290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-fa4d2b98-2a83-4530-b6fc-b4b6849eb604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:16.443366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 328/659] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:16.444239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d2b98-2a83-4530-b6fc-b4b6849eb604 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:16.475934 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79518a84-0e64-4aa0-bb49-c59d15f252bd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:16.533231 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545e794-ae34-49db-a3d3-a68ef06cd850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:16.534204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545e794-ae34-49db-a3d3-a68ef06cd850 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:16.534486 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545e794-ae34-49db-a3d3-a68ef06cd850 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:16.535723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545e794-ae34-49db-a3d3-a68ef06cd850 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.535723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545e794-ae34-49db-a3d3-a68ef06cd850 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:16.535914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545e794-ae34-49db-a3d3-a68ef06cd850 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:16.538934 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545e794-ae34-49db-a3d3-a68ef06cd850 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:16.560945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 332/660] 10.222.0.55 () {56 vars in 1137 bytes} [Fri Aug 19 16:19:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:16.560945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545e794-ae34-49db-a3d3-a68ef06cd850 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:16.567442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:16.582094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d2b98-2a83-4530-b6fc-b4b6849eb604 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:16.599032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:16.626143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f153f02-e4c3-4bb7-8339-44d007f7bcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:16.627269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f153f02-e4c3-4bb7-8339-44d007f7bcae None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:16.627558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f153f02-e4c3-4bb7-8339-44d007f7bcae None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:16.627836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f153f02-e4c3-4bb7-8339-44d007f7bcae None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.628485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f153f02-e4c3-4bb7-8339-44d007f7bcae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:16.636835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f153f02-e4c3-4bb7-8339-44d007f7bcae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:16.636835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f153f02-e4c3-4bb7-8339-44d007f7bcae None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:16.655949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 329/661] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:16.656946 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f153f02-e4c3-4bb7-8339-44d007f7bcae None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:16.695704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545e794-ae34-49db-a3d3-a68ef06cd850 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:16.738723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10d4fb-6e23-447e-bb85-4c1aa18afe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:16.739450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10d4fb-6e23-447e-bb85-4c1aa18afe2b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:16.739730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10d4fb-6e23-447e-bb85-4c1aa18afe2b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:16.739998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10d4fb-6e23-447e-bb85-4c1aa18afe2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.746616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10d4fb-6e23-447e-bb85-4c1aa18afe2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:16.748250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10d4fb-6e23-447e-bb85-4c1aa18afe2b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-ab10d4fb-6e23-447e-bb85-4c1aa18afe2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:16.759686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:16.760612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:16.761533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:16.761533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:16.761533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:16.761533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:16.761533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:16.761533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:16.761533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:16.761533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:16.765371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 333/662] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:16.766134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10d4fb-6e23-447e-bb85-4c1aa18afe2b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:16.776019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f153f02-e4c3-4bb7-8339-44d007f7bcae None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:16.812647 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb879cfa-0d7f-4dd7-92ef-6a09e9ecf127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:16.813194 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb879cfa-0d7f-4dd7-92ef-6a09e9ecf127 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:16.813477 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb879cfa-0d7f-4dd7-92ef-6a09e9ecf127 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:16.813751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb879cfa-0d7f-4dd7-92ef-6a09e9ecf127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.829842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:16.860016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:16.873856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10d4fb-6e23-447e-bb85-4c1aa18afe2b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:16.888430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb879cfa-0d7f-4dd7-92ef-6a09e9ecf127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:16.889945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb879cfa-0d7f-4dd7-92ef-6a09e9ecf127 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:16.913936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e317ab-166b-4a13-a740-8e34526193c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:16.914862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e317ab-166b-4a13-a740-8e34526193c9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:16.915330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e317ab-166b-4a13-a740-8e34526193c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:16.915605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e317ab-166b-4a13-a740-8e34526193c9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:16.922986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e317ab-166b-4a13-a740-8e34526193c9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:16.924513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e317ab-166b-4a13-a740-8e34526193c9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-11e317ab-166b-4a13-a740-8e34526193c9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:16.936834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:16.937814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:16.938760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:16.938760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:16.938760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:16.938760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:16.938760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:16.938760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:16.938760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:16.938760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:16.938760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 16:19:16.938760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:16.938760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 334/663] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:16.938760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e317ab-166b-4a13-a740-8e34526193c9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:17.004425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 330/664] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:19:17.004425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb879cfa-0d7f-4dd7-92ef-6a09e9ecf127 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:17.049914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e317ab-166b-4a13-a740-8e34526193c9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:17.078146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:17.078311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:17.090708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab5461-9d6c-4afe-9e14-49b7453a9553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:17.091345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab5461-9d6c-4afe-9e14-49b7453a9553 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:17.091621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab5461-9d6c-4afe-9e14-49b7453a9553 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:17.091895 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab5461-9d6c-4afe-9e14-49b7453a9553 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:17.102433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab5461-9d6c-4afe-9e14-49b7453a9553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-59ab5461-9d6c-4afe-9e14-49b7453a9553 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:17.107804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:17.108716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:17.109609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 335/665] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:17.110289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab5461-9d6c-4afe-9e14-49b7453a9553 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:17.116538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb879cfa-0d7f-4dd7-92ef-6a09e9ecf127 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:17.153564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dd122-dbc6-46e1-b89e-549b6faf141f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:17.154174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dd122-dbc6-46e1-b89e-549b6faf141f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:17.154460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dd122-dbc6-46e1-b89e-549b6faf141f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:17.154734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dd122-dbc6-46e1-b89e-549b6faf141f None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:17.155417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dd122-dbc6-46e1-b89e-549b6faf141f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:17.155721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dd122-dbc6-46e1-b89e-549b6faf141f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:17.157188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dd122-dbc6-46e1-b89e-549b6faf141f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:17.169236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 331/666] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:17.169898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dd122-dbc6-46e1-b89e-549b6faf141f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:17.223062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab5461-9d6c-4afe-9e14-49b7453a9553 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:17.259572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04e9b0-434e-45a7-a0f6-082e1b5e68c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:17.260193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04e9b0-434e-45a7-a0f6-082e1b5e68c0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:17.260507 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04e9b0-434e-45a7-a0f6-082e1b5e68c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:17.260832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04e9b0-434e-45a7-a0f6-082e1b5e68c0 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:17.261581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04e9b0-434e-45a7-a0f6-082e1b5e68c0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:17.261918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04e9b0-434e-45a7-a0f6-082e1b5e68c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:17.263369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04e9b0-434e-45a7-a0f6-082e1b5e68c0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:17.272481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dd122-dbc6-46e1-b89e-549b6faf141f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:17.285457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 336/667] 10.222.0.55 () {56 vars in 1139 bytes} [Fri Aug 19 16:19:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:17.285457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04e9b0-434e-45a7-a0f6-082e1b5e68c0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:17.309241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:17.315299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf000ac-1e73-4c6d-99b4-b059fe640b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:17.315934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf000ac-1e73-4c6d-99b4-b059fe640b8d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:17.316217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf000ac-1e73-4c6d-99b4-b059fe640b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:17.316490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf000ac-1e73-4c6d-99b4-b059fe640b8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:17.326307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:17.329145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf000ac-1e73-4c6d-99b4-b059fe640b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:17.330535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf000ac-1e73-4c6d-99b4-b059fe640b8d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-5bf000ac-1e73-4c6d-99b4-b059fe640b8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:17.345568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:17.346526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:17.347429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:17.347429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:17.347429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:17.347429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 332/668] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:17.347429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf000ac-1e73-4c6d-99b4-b059fe640b8d None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:17.393229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04e9b0-434e-45a7-a0f6-082e1b5e68c0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:17.430904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb27e54-a4f5-4884-b384-b1bf135b7b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:17.431528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb27e54-a4f5-4884-b384-b1bf135b7b27 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:17.431838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb27e54-a4f5-4884-b384-b1bf135b7b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:17.432139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb27e54-a4f5-4884-b384-b1bf135b7b27 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:17.432861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb27e54-a4f5-4884-b384-b1bf135b7b27 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:17.433190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb27e54-a4f5-4884-b384-b1bf135b7b27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:17.434579 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb27e54-a4f5-4884-b384-b1bf135b7b27 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:17.448108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 337/669] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:17.448108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb27e54-a4f5-4884-b384-b1bf135b7b27 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:17.463265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf000ac-1e73-4c6d-99b4-b059fe640b8d None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:17.500655 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5667f3d-8b09-4664-b1de-08cc31494f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:17.501416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5667f3d-8b09-4664-b1de-08cc31494f8c None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:17.501698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5667f3d-8b09-4664-b1de-08cc31494f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:17.501969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5667f3d-8b09-4664-b1de-08cc31494f8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:17.502685 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5667f3d-8b09-4664-b1de-08cc31494f8c None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=748ad823b944441d907994cda6d8ac7f) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:17.502990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5667f3d-8b09-4664-b1de-08cc31494f8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:17.504416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5667f3d-8b09-4664-b1de-08cc31494f8c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:17.520789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 333/670] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:17 2022] GET /identity/v3/role_assignments?user.id=748ad823b944441d907994cda6d8ac7f&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:17.520789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5667f3d-8b09-4664-b1de-08cc31494f8c None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:17.581221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:17.594335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb27e54-a4f5-4884-b384-b1bf135b7b27 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:17.596065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:17.634781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af88cf-72bd-4ff7-84d2-e724dc671cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:17.635314 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af88cf-72bd-4ff7-84d2-e724dc671cec None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:17.635640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af88cf-72bd-4ff7-84d2-e724dc671cec None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:17.635951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af88cf-72bd-4ff7-84d2-e724dc671cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:17.662412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5667f3d-8b09-4664-b1de-08cc31494f8c None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:17.701083 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780d0fd-d9da-47a3-946d-27cfc5f7285c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:17.701744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780d0fd-d9da-47a3-946d-27cfc5f7285c None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:17.701981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780d0fd-d9da-47a3-946d-27cfc5f7285c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:17.702254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780d0fd-d9da-47a3-946d-27cfc5f7285c None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:17.702896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780d0fd-d9da-47a3-946d-27cfc5f7285c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:17.703196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780d0fd-d9da-47a3-946d-27cfc5f7285c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:17.704576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780d0fd-d9da-47a3-946d-27cfc5f7285c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:17.721405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 334/671] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:17.722318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780d0fd-d9da-47a3-946d-27cfc5f7285c None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:17.723653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af88cf-72bd-4ff7-84d2-e724dc671cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:17.726506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af88cf-72bd-4ff7-84d2-e724dc671cec None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:17.851722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780d0fd-d9da-47a3-946d-27cfc5f7285c None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:17.861409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:17.883655 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 338/672] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:19:17.885451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af88cf-72bd-4ff7-84d2-e724dc671cec None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:17.887490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c2a9d-da15-459a-970f-278831ef3aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:17.888198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c2a9d-da15-459a-970f-278831ef3aeb None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:17.888545 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c2a9d-da15-459a-970f-278831ef3aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:17.888918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c2a9d-da15-459a-970f-278831ef3aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:17.889862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:17.901020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c2a9d-da15-459a-970f-278831ef3aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:17.902209 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c2a9d-da15-459a-970f-278831ef3aeb None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-594c2a9d-da15-459a-970f-278831ef3aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:17.917722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:17.918767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:17.919720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:17.919720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:17.919720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:17.919720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 335/673] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:17.919956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c2a9d-da15-459a-970f-278831ef3aeb None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:18.026928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af88cf-72bd-4ff7-84d2-e724dc671cec None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:18.027150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c2a9d-da15-459a-970f-278831ef3aeb None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:18.062042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae7a26-ceda-4d7e-bcc7-5c92b7e64ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:18.062672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae7a26-ceda-4d7e-bcc7-5c92b7e64ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.062949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae7a26-ceda-4d7e-bcc7-5c92b7e64ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.063217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae7a26-ceda-4d7e-bcc7-5c92b7e64ee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:18.071459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785e21e-3aa3-4f6b-857a-c8cc8b701994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:18.072538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785e21e-3aa3-4f6b-857a-c8cc8b701994 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.073042 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785e21e-3aa3-4f6b-857a-c8cc8b701994 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.073575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785e21e-3aa3-4f6b-857a-c8cc8b701994 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:18.075539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae7a26-ceda-4d7e-bcc7-5c92b7e64ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:18.077183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae7a26-ceda-4d7e-bcc7-5c92b7e64ee6 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:18.081811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785e21e-3aa3-4f6b-857a-c8cc8b701994 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:18.084853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785e21e-3aa3-4f6b-857a-c8cc8b701994 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-85ae7a26-ceda-4d7e-bcc7-5c92b7e64ee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:18.092106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:18.093061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:18.093936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:18.093936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:18.093936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:18.093936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 336/674] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:18.094450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae7a26-ceda-4d7e-bcc7-5c92b7e64ee6 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c785e21e-3aa3-4f6b-857a-c8cc8b701994 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:18.101233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:18.102390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:18.103613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:18.103613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:18.103613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:18.103613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:18.103613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:18.103613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:18.103613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:18.103613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:18.105740 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 339/675] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:18.106833 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785e21e-3aa3-4f6b-857a-c8cc8b701994 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:18.110716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:18.182040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:18.199114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae7a26-ceda-4d7e-bcc7-5c92b7e64ee6 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:18.228718 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785e21e-3aa3-4f6b-857a-c8cc8b701994 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:18.234084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea607360-8019-4b40-9837-58e30b004fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:18.234694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607360-8019-4b40-9837-58e30b004fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.234972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607360-8019-4b40-9837-58e30b004fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.235239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607360-8019-4b40-9837-58e30b004fd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:18.235876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea607360-8019-4b40-9837-58e30b004fd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:18.236174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea607360-8019-4b40-9837-58e30b004fd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:18.237579 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea607360-8019-4b40-9837-58e30b004fd8 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:18.254474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 337/676] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:18.255910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea607360-8019-4b40-9837-58e30b004fd8 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:18.268188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d57a5-872d-4b3c-aaef-523a6e5baa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:18.269105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d57a5-872d-4b3c-aaef-523a6e5baa75 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.269577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d57a5-872d-4b3c-aaef-523a6e5baa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.270026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d57a5-872d-4b3c-aaef-523a6e5baa75 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:18.271713 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d57a5-872d-4b3c-aaef-523a6e5baa75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:18.272211 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d57a5-872d-4b3c-aaef-523a6e5baa75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:18.273820 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d57a5-872d-4b3c-aaef-523a6e5baa75 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:18.286383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 340/677] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:18.287602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d57a5-872d-4b3c-aaef-523a6e5baa75 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:18.328978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:18.361532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea607360-8019-4b40-9837-58e30b004fd8 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:18.395326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d57a5-872d-4b3c-aaef-523a6e5baa75 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:18.398075 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf2aa3-c1e7-4c00-aa25-72750c681ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:18.398679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf2aa3-c1e7-4c00-aa25-72750c681ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.398955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf2aa3-c1e7-4c00-aa25-72750c681ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.399223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf2aa3-c1e7-4c00-aa25-72750c681ff5 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:18.399905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf2aa3-c1e7-4c00-aa25-72750c681ff5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:18.400205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf2aa3-c1e7-4c00-aa25-72750c681ff5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:18.401678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf2aa3-c1e7-4c00-aa25-72750c681ff5 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:18.413392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 338/678] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:18.414162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf2aa3-c1e7-4c00-aa25-72750c681ff5 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:18.423800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:18.437436 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed8055-0a1e-46c3-be4d-c975a91115a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:18.438066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed8055-0a1e-46c3-be4d-c975a91115a6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.438344 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed8055-0a1e-46c3-be4d-c975a91115a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.438613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed8055-0a1e-46c3-be4d-c975a91115a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:18.446887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed8055-0a1e-46c3-be4d-c975a91115a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:18.448306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed8055-0a1e-46c3-be4d-c975a91115a6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-e4ed8055-0a1e-46c3-be4d-c975a91115a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:18.459162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:18.460094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 341/679] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:19:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d822fa-d362-4868-8a0a-4bb03ff6a50d None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.461598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d822fa-d362-4868-8a0a-4bb03ff6a50d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.462345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d822fa-d362-4868-8a0a-4bb03ff6a50d None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:18.467476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 342/680] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:18.467476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d822fa-d362-4868-8a0a-4bb03ff6a50d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:18.528261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf2aa3-c1e7-4c00-aa25-72750c681ff5 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:18.558848 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:18.567245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed6de9-21ce-4091-aa70-33d0114b71dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:18.568442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed6de9-21ce-4091-aa70-33d0114b71dd None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.568702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed6de9-21ce-4091-aa70-33d0114b71dd None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.569029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed6de9-21ce-4091-aa70-33d0114b71dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:18.575792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed6de9-21ce-4091-aa70-33d0114b71dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:18.577242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed6de9-21ce-4091-aa70-33d0114b71dd None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:18.579034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d822fa-d362-4868-8a0a-4bb03ff6a50d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-70ed6de9-21ce-4091-aa70-33d0114b71dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:18.588293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:18.589235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:18.590739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:18.590739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:18.590739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:18.590739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:18.590739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:18.590739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:18.590739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:18.590739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:18.590739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 339/681] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:18.590739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed6de9-21ce-4091-aa70-33d0114b71dd None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:18.621704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f940f3-5d87-4db9-bab9-de56cc08c449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:18.622233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f940f3-5d87-4db9-bab9-de56cc08c449 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.622520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f940f3-5d87-4db9-bab9-de56cc08c449 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.622811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f940f3-5d87-4db9-bab9-de56cc08c449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:18.623507 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f940f3-5d87-4db9-bab9-de56cc08c449 None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=1eaa3980ad8840879e4e426b00e3c56f) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:18.623813 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f940f3-5d87-4db9-bab9-de56cc08c449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:18.625318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f940f3-5d87-4db9-bab9-de56cc08c449 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:18.632395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 343/682] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:18 2022] GET /identity/v3/role_assignments?user.id=1eaa3980ad8840879e4e426b00e3c56f&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:18.633419 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ec149-794e-445c-bc51-24e10f6aba27 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.633703 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ec149-794e-445c-bc51-24e10f6aba27 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.633976 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ec149-794e-445c-bc51-24e10f6aba27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:18.658023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0e1ec149-794e-445c-bc51-24e10f6aba27 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:18.659583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0e1ec149-794e-445c-bc51-24e10f6aba27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:18.717033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed6de9-21ce-4091-aa70-33d0114b71dd None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:18.722191 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:18.761776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 344/683] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:18.763063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ec149-794e-445c-bc51-24e10f6aba27 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:18.763982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c89b5-502b-47c8-861e-bc61b0b80237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:18.764618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c89b5-502b-47c8-861e-bc61b0b80237 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.764927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c89b5-502b-47c8-861e-bc61b0b80237 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.765198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c89b5-502b-47c8-861e-bc61b0b80237 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:18.765958 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c89b5-502b-47c8-861e-bc61b0b80237 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:18.766264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c89b5-502b-47c8-861e-bc61b0b80237 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:18.773059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c89b5-502b-47c8-861e-bc61b0b80237 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:18.795250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 340/684] 10.222.0.55 () {56 vars in 1135 bytes} [Fri Aug 19 16:19:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:18.803095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c89b5-502b-47c8-861e-bc61b0b80237 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:18.873784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:18.898456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ec149-794e-445c-bc51-24e10f6aba27 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:18.936485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c89b5-502b-47c8-861e-bc61b0b80237 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:18.939868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672ad50-9ca4-47cb-bbe9-36094bc4ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:18.940735 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672ad50-9ca4-47cb-bbe9-36094bc4ee40 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.941275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672ad50-9ca4-47cb-bbe9-36094bc4ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.941773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672ad50-9ca4-47cb-bbe9-36094bc4ee40 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:18.942719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672ad50-9ca4-47cb-bbe9-36094bc4ee40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:18.943277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672ad50-9ca4-47cb-bbe9-36094bc4ee40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:18.945068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672ad50-9ca4-47cb-bbe9-36094bc4ee40 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:18.963063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 345/685] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:18.964232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672ad50-9ca4-47cb-bbe9-36094bc4ee40 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:18.975632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854456f-dc9f-4aa8-ac5d-cf915f952c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:18.976254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854456f-dc9f-4aa8-ac5d-cf915f952c93 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:18.976532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854456f-dc9f-4aa8-ac5d-cf915f952c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:18.976824 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854456f-dc9f-4aa8-ac5d-cf915f952c93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:19.017137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:19.084619 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672ad50-9ca4-47cb-bbe9-36094bc4ee40 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:19.105263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854456f-dc9f-4aa8-ac5d-cf915f952c93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:19.107167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854456f-dc9f-4aa8-ac5d-cf915f952c93 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:19.126156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ea8ad-fe47-462a-8451-c6a03add0679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:19.126763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ea8ad-fe47-462a-8451-c6a03add0679 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:19.127045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ea8ad-fe47-462a-8451-c6a03add0679 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:19.127314 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ea8ad-fe47-462a-8451-c6a03add0679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:19.128116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ea8ad-fe47-462a-8451-c6a03add0679 None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=9e0b7efc488a42ffab8ba7e2d101ced4) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:19.128425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ea8ad-fe47-462a-8451-c6a03add0679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:19.130202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ea8ad-fe47-462a-8451-c6a03add0679 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:19.137047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 346/686] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:18 2022] GET /identity/v3/role_assignments?user.id=9e0b7efc488a42ffab8ba7e2d101ced4&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:19.137920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ea8ad-fe47-462a-8451-c6a03add0679 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:19.195278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:19.312002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 341/687] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:19:19.316909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:19.317800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854456f-dc9f-4aa8-ac5d-cf915f952c93 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:19.323212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ea8ad-fe47-462a-8451-c6a03add0679 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:19.366367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd6c59-2106-41bf-90e9-1d969eab6a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:19.367011 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6c59-2106-41bf-90e9-1d969eab6a35 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:19.367297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6c59-2106-41bf-90e9-1d969eab6a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:19.367573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6c59-2106-41bf-90e9-1d969eab6a35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:19.380031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd6c59-2106-41bf-90e9-1d969eab6a35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a1fd6c59-2106-41bf-90e9-1d969eab6a35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:19.386005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:19.386966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:19.387793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 347/688] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:19:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:19.392743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd6c59-2106-41bf-90e9-1d969eab6a35 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:19.449524 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854456f-dc9f-4aa8-ac5d-cf915f952c93 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:19.497295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61809634-07ea-4cf3-bc5e-84ab6256bb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:19.498489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61809634-07ea-4cf3-bc5e-84ab6256bb07 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:19.498970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61809634-07ea-4cf3-bc5e-84ab6256bb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:19.499248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61809634-07ea-4cf3-bc5e-84ab6256bb07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:19.512467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61809634-07ea-4cf3-bc5e-84ab6256bb07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:19.514814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61809634-07ea-4cf3-bc5e-84ab6256bb07 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:19.530808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd6c59-2106-41bf-90e9-1d969eab6a35 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-61809634-07ea-4cf3-bc5e-84ab6256bb07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:19.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:19.540601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:19.541556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:19.541556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:19.541556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:19.544861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 342/689] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:19.546558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61809634-07ea-4cf3-bc5e-84ab6256bb07 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:19.549622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:19.576326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b928e-4b21-460d-bded-74e3be2c9d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:19.577763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b928e-4b21-460d-bded-74e3be2c9d5c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:19.578265 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b928e-4b21-460d-bded-74e3be2c9d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:19.578744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b928e-4b21-460d-bded-74e3be2c9d5c None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:19.579612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b928e-4b21-460d-bded-74e3be2c9d5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:19.580604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b928e-4b21-460d-bded-74e3be2c9d5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:19.582227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b928e-4b21-460d-bded-74e3be2c9d5c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:19.593195 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 348/690] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:19.599049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b928e-4b21-460d-bded-74e3be2c9d5c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:19.662319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:19.721276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61809634-07ea-4cf3-bc5e-84ab6256bb07 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:19.753621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b928e-4b21-460d-bded-74e3be2c9d5c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:19.762282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095b33b-6285-4b7e-b25d-bf2bb29db9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:19.762899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095b33b-6285-4b7e-b25d-bf2bb29db9ac None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:19.763178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095b33b-6285-4b7e-b25d-bf2bb29db9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:19.763451 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095b33b-6285-4b7e-b25d-bf2bb29db9ac None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:19.764094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095b33b-6285-4b7e-b25d-bf2bb29db9ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:19.764396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095b33b-6285-4b7e-b25d-bf2bb29db9ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:19.765795 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095b33b-6285-4b7e-b25d-bf2bb29db9ac None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:19.778740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 343/691] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:19.779948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cb517-8090-42de-93cd-b3a5f4d15708 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:19.780235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cb517-8090-42de-93cd-b3a5f4d15708 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:19.780505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cb517-8090-42de-93cd-b3a5f4d15708 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:19.781979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 344/692] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:19.784968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cb517-8090-42de-93cd-b3a5f4d15708 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:19.794793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cd874-de73-4a28-8c5d-78c1c1626d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:19.795428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cd874-de73-4a28-8c5d-78c1c1626d50 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:19.795706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cd874-de73-4a28-8c5d-78c1c1626d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:19.796067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cd874-de73-4a28-8c5d-78c1c1626d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:19.808116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cd874-de73-4a28-8c5d-78c1c1626d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:19.809728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cd874-de73-4a28-8c5d-78c1c1626d50 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-ed3cd874-de73-4a28-8c5d-78c1c1626d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:19.826572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:19.827559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:19.829644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:19.829644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:19.829644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:19.829644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 349/693] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:19.829644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e610f5-648e-451a-9129-ee46e250706f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:19.829644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e610f5-648e-451a-9129-ee46e250706f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:19.830431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e610f5-648e-451a-9129-ee46e250706f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:19.856652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-76e610f5-648e-451a-9129-ee46e250706f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:19.857896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-76e610f5-648e-451a-9129-ee46e250706f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:19.860892 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:19.904066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cb517-8090-42de-93cd-b3a5f4d15708 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:19.944077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 350/694] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:19.944978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e610f5-648e-451a-9129-ee46e250706f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:19.950843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:19.957181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e128b-42d9-4a3a-9473-82e203f87840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:19.957839 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e128b-42d9-4a3a-9473-82e203f87840 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:19.958279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e128b-42d9-4a3a-9473-82e203f87840 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:19.958741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e128b-42d9-4a3a-9473-82e203f87840 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:19.973972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e128b-42d9-4a3a-9473-82e203f87840 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:19.975437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e128b-42d9-4a3a-9473-82e203f87840 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-719e128b-42d9-4a3a-9473-82e203f87840 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:20.000780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:20.001706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:20.002569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:20.002569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:20.002569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:20.002569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:20.002569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:20.002569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:20.002569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:20.002569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:20.005625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 345/695] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:20.011396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e128b-42d9-4a3a-9473-82e203f87840 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:20.107938 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e610f5-648e-451a-9129-ee46e250706f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:20.150903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6aa5-1e6d-4d3e-baa2-722d5f211dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:20.151529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6aa5-1e6d-4d3e-baa2-722d5f211dfd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:20.151814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6aa5-1e6d-4d3e-baa2-722d5f211dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:20.152089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6aa5-1e6d-4d3e-baa2-722d5f211dfd None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:20.152734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c6aa5-1e6d-4d3e-baa2-722d5f211dfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:20.153281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c6aa5-1e6d-4d3e-baa2-722d5f211dfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:20.154670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c6aa5-1e6d-4d3e-baa2-722d5f211dfd None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:20.180371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 351/696] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:20.185660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6aa5-1e6d-4d3e-baa2-722d5f211dfd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:20.222917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e128b-42d9-4a3a-9473-82e203f87840 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:20.253219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:20.278380 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:20.294014 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc8621-64e6-4d69-b70e-8a4452c8e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:20.294617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc8621-64e6-4d69-b70e-8a4452c8e66d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:20.294893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc8621-64e6-4d69-b70e-8a4452c8e66d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:20.295162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc8621-64e6-4d69-b70e-8a4452c8e66d None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:20.295850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc8621-64e6-4d69-b70e-8a4452c8e66d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:20.296041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6aa5-1e6d-4d3e-baa2-722d5f211dfd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:20.296164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc8621-64e6-4d69-b70e-8a4452c8e66d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:20.297685 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc8621-64e6-4d69-b70e-8a4452c8e66d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:20.311395 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 346/697] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:20.312430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc8621-64e6-4d69-b70e-8a4452c8e66d None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:20.338349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da167821-0cd1-4f55-a132-c91184e9eb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:20.339319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da167821-0cd1-4f55-a132-c91184e9eb2f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:20.339611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da167821-0cd1-4f55-a132-c91184e9eb2f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:20.340025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da167821-0cd1-4f55-a132-c91184e9eb2f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:20.350512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da167821-0cd1-4f55-a132-c91184e9eb2f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:20.353131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da167821-0cd1-4f55-a132-c91184e9eb2f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-da167821-0cd1-4f55-a132-c91184e9eb2f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:20.367564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:20.369993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:20.372358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:20.372358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:20.372358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:20.372358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:20.372358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:20.372358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:20.372358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:20.372358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:20.372358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:19:20.372358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:20.372358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 352/698] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Aug 19 16:19:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:20.372358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da167821-0cd1-4f55-a132-c91184e9eb2f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:20.457899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc8621-64e6-4d69-b70e-8a4452c8e66d None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:20.494245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da167821-0cd1-4f55-a132-c91184e9eb2f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:20.501756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9619d3e-0857-4acd-a38f-698d3da2715c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:20.503201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9619d3e-0857-4acd-a38f-698d3da2715c None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:20.503709 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9619d3e-0857-4acd-a38f-698d3da2715c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:20.504195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9619d3e-0857-4acd-a38f-698d3da2715c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:20.518307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9619d3e-0857-4acd-a38f-698d3da2715c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:20.520714 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9619d3e-0857-4acd-a38f-698d3da2715c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:20.521743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:20.539482 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dc8a1-184b-4b45-b5cf-72d9ba39ef49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:20.540097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dc8a1-184b-4b45-b5cf-72d9ba39ef49 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:20.540381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dc8a1-184b-4b45-b5cf-72d9ba39ef49 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:20.540657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dc8a1-184b-4b45-b5cf-72d9ba39ef49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:20.541414 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dc8a1-184b-4b45-b5cf-72d9ba39ef49 None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=c755f9538e984bc3877112896595e158) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:20.541725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dc8a1-184b-4b45-b5cf-72d9ba39ef49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:20.543349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dc8a1-184b-4b45-b5cf-72d9ba39ef49 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-c9619d3e-0857-4acd-a38f-698d3da2715c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:20.544265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:20.545282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:20.546180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:20.546180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:20.546180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:20.548910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 347/699] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:20.550309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9619d3e-0857-4acd-a38f-698d3da2715c None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:20.560923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 353/700] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:20 2022] GET /identity/v3/role_assignments?user.id=c755f9538e984bc3877112896595e158&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:20.560923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dc8a1-184b-4b45-b5cf-72d9ba39ef49 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:20.626389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:20.754365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9619d3e-0857-4acd-a38f-698d3da2715c None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:20.759341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dc8a1-184b-4b45-b5cf-72d9ba39ef49 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:20.793757 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdb40d-79de-4bd5-891d-f168c61e9b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:20.794684 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdb40d-79de-4bd5-891d-f168c61e9b1d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:20.795172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdb40d-79de-4bd5-891d-f168c61e9b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:20.795640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdb40d-79de-4bd5-891d-f168c61e9b1d None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:20.796562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafdb40d-79de-4bd5-891d-f168c61e9b1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:20.797106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafdb40d-79de-4bd5-891d-f168c61e9b1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:20.798853 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafdb40d-79de-4bd5-891d-f168c61e9b1d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:20.809526 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde0980-1bc8-4b75-863a-af10699e77d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:20.810155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde0980-1bc8-4b75-863a-af10699e77d1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:20.810440 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde0980-1bc8-4b75-863a-af10699e77d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:20.810716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde0980-1bc8-4b75-863a-af10699e77d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:20.811371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde0980-1bc8-4b75-863a-af10699e77d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:20.811676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde0980-1bc8-4b75-863a-af10699e77d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:20.813746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde0980-1bc8-4b75-863a-af10699e77d1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:20.822698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 348/701] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Aug 19 16:19:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:20.828638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdb40d-79de-4bd5-891d-f168c61e9b1d None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:20.829693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 354/702] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:20.837750 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde0980-1bc8-4b75-863a-af10699e77d1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:20.895074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:20.929489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:20.939816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdb40d-79de-4bd5-891d-f168c61e9b1d None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:20.950870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde0980-1bc8-4b75-863a-af10699e77d1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:20.977776 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb240c10-0ec8-4a6e-a049-2b04ddc79e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:20.978532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb240c10-0ec8-4a6e-a049-2b04ddc79e35 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:20.978883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb240c10-0ec8-4a6e-a049-2b04ddc79e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:20.979159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb240c10-0ec8-4a6e-a049-2b04ddc79e35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:20.992028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fce14-5d3d-49d0-bbdd-6a4b7098c3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:20.992644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fce14-5d3d-49d0-bbdd-6a4b7098c3ef None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:20.992957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fce14-5d3d-49d0-bbdd-6a4b7098c3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:20.993277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fce14-5d3d-49d0-bbdd-6a4b7098c3ef None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:20.993932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fce14-5d3d-49d0-bbdd-6a4b7098c3ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:20.995397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fce14-5d3d-49d0-bbdd-6a4b7098c3ef None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:21.027323 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 355/703] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:19:20 2022] POST /identity/v3/users => generated 254 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:21.028932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13b4fc-8a16-421a-a6fe-4881ae4ac6bc None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:21.029429 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13b4fc-8a16-421a-a6fe-4881ae4ac6bc None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:21.029924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13b4fc-8a16-421a-a6fe-4881ae4ac6bc None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:21.031704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 356/704] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:21.038036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048bcc8-516b-4769-bccd-4c3749c0414a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:21.038036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048bcc8-516b-4769-bccd-4c3749c0414a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:21.038292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048bcc8-516b-4769-bccd-4c3749c0414a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:21.061005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c048bcc8-516b-4769-bccd-4c3749c0414a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:21.062369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c048bcc8-516b-4769-bccd-4c3749c0414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:21.073364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb240c10-0ec8-4a6e-a049-2b04ddc79e35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:21.076017 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb240c10-0ec8-4a6e-a049-2b04ddc79e35 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:21.146738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 357/705] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:19:21.197335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:21.208143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 349/706] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:19:21.213269 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048bcc8-516b-4769-bccd-4c3749c0414a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:21.263692 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:21.351552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048bcc8-516b-4769-bccd-4c3749c0414a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:21.387621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e85fc9-b9d8-4b09-bcf1-e0fadd6c85da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:21.388257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e85fc9-b9d8-4b09-bcf1-e0fadd6c85da None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:21.388533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e85fc9-b9d8-4b09-bcf1-e0fadd6c85da None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:21.388827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e85fc9-b9d8-4b09-bcf1-e0fadd6c85da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:21.399395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e85fc9-b9d8-4b09-bcf1-e0fadd6c85da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:21.400862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e85fc9-b9d8-4b09-bcf1-e0fadd6c85da None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-79e85fc9-b9d8-4b09-bcf1-e0fadd6c85da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:21.416597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:21.417575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:21.418456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:21.418456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:21.418456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:21.418456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 358/707] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:21.423494 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb240c10-0ec8-4a6e-a049-2b04ddc79e35 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:21.449396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e85fc9-b9d8-4b09-bcf1-e0fadd6c85da None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:21.544026 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb240c10-0ec8-4a6e-a049-2b04ddc79e35 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:21.564153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e85fc9-b9d8-4b09-bcf1-e0fadd6c85da None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:21.571673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:21.581736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b1d44-e000-4bf6-af71-9042c2f466a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:21.582350 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b1d44-e000-4bf6-af71-9042c2f466a8 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:21.582633 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b1d44-e000-4bf6-af71-9042c2f466a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:21.583434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b1d44-e000-4bf6-af71-9042c2f466a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:21.584260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b1d44-e000-4bf6-af71-9042c2f466a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:21.584572 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b1d44-e000-4bf6-af71-9042c2f466a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:21.586840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b1d44-e000-4bf6-af71-9042c2f466a8 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:21.600729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 350/708] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:21.601039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b32cdf-8727-46f9-a5da-0248d422dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:21.601700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b32cdf-8727-46f9-a5da-0248d422dcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:21.601923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b32cdf-8727-46f9-a5da-0248d422dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:21.602024 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea30f8-23d2-4f8b-ba95-38276864e800 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:21.602201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b32cdf-8727-46f9-a5da-0248d422dcb4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:21.602295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea30f8-23d2-4f8b-ba95-38276864e800 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:21.602816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea30f8-23d2-4f8b-ba95-38276864e800 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:21.604271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 351/709] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:21.605202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b32cdf-8727-46f9-a5da-0248d422dcb4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:21.606604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b32cdf-8727-46f9-a5da-0248d422dcb4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:21.607777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea30f8-23d2-4f8b-ba95-38276864e800 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-57b32cdf-8727-46f9-a5da-0248d422dcb4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:21.627097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:21.628183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 16:19:21.629036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:21.630571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 359/710] 10.222.0.55 () {56 vars in 1129 bytes} [Fri Aug 19 16:19:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:21.632534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6f17d-eaa8-416c-aa53-0ab8d88cb2aa None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:21.632846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6f17d-eaa8-416c-aa53-0ab8d88cb2aa None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:21.633178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6f17d-eaa8-416c-aa53-0ab8d88cb2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:21.658745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ecf6f17d-eaa8-416c-aa53-0ab8d88cb2aa None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:21.659977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ecf6f17d-eaa8-416c-aa53-0ab8d88cb2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:21.694136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:21.740477 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 360/711] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:21.741448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6f17d-eaa8-416c-aa53-0ab8d88cb2aa None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:21.762461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea30f8-23d2-4f8b-ba95-38276864e800 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:21.800734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bafcb-867f-4b5f-a985-1194cf3bff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:21.801943 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bafcb-867f-4b5f-a985-1194cf3bff22 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:21.802439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bafcb-867f-4b5f-a985-1194cf3bff22 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:21.802907 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bafcb-867f-4b5f-a985-1194cf3bff22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:21.811026 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bafcb-867f-4b5f-a985-1194cf3bff22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:21.812690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bafcb-867f-4b5f-a985-1194cf3bff22 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-a00bafcb-867f-4b5f-a985-1194cf3bff22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:21.827474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:21.828410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:21.829379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:21.829379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:21.829379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:21.829379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:21.829379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:21.829379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:21.829379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:21.829379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:21.831558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 352/712] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:21.838016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bafcb-867f-4b5f-a985-1194cf3bff22 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:21.914176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:21.951680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bafcb-867f-4b5f-a985-1194cf3bff22 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:21.977110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:21.986353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6f17d-eaa8-416c-aa53-0ab8d88cb2aa None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:21.989295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc1191-4d61-4ac4-a929-c4d644b74177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:21.991117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc1191-4d61-4ac4-a929-c4d644b74177 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:21.991601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc1191-4d61-4ac4-a929-c4d644b74177 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:21.992070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc1191-4d61-4ac4-a929-c4d644b74177 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:21.992996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc1191-4d61-4ac4-a929-c4d644b74177 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:21.993517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc1191-4d61-4ac4-a929-c4d644b74177 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:21.995145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc1191-4d61-4ac4-a929-c4d644b74177 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:22.009671 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 353/713] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:22.015805 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc1191-4d61-4ac4-a929-c4d644b74177 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:22.070850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f772d-89de-4b9e-be15-8ea344d1b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:22.071232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f772d-89de-4b9e-be15-8ea344d1b46d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:22.071443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f772d-89de-4b9e-be15-8ea344d1b46d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:22.071720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f772d-89de-4b9e-be15-8ea344d1b46d None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:22.072362 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f772d-89de-4b9e-be15-8ea344d1b46d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:22.072669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f772d-89de-4b9e-be15-8ea344d1b46d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:22.074167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f772d-89de-4b9e-be15-8ea344d1b46d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:22.084602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 361/714] 10.222.0.55 () {56 vars in 1124 bytes} [Fri Aug 19 16:19:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:22.085560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f772d-89de-4b9e-be15-8ea344d1b46d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:22.125839 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc1191-4d61-4ac4-a929-c4d644b74177 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:22.163149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d09a7f-3da8-43e1-aeb4-94879ce16f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:22.164152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d09a7f-3da8-43e1-aeb4-94879ce16f05 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:22.164682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d09a7f-3da8-43e1-aeb4-94879ce16f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:22.165184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d09a7f-3da8-43e1-aeb4-94879ce16f05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:22.166094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d09a7f-3da8-43e1-aeb4-94879ce16f05 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=4739e80619a54b178b9aacec6a42a7f0) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:22.166595 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d09a7f-3da8-43e1-aeb4-94879ce16f05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:22.168227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d09a7f-3da8-43e1-aeb4-94879ce16f05 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:22.174312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 354/715] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:22 2022] GET /identity/v3/role_assignments?user.id=4739e80619a54b178b9aacec6a42a7f0&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:22.181911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d09a7f-3da8-43e1-aeb4-94879ce16f05 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:22.203846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f772d-89de-4b9e-be15-8ea344d1b46d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:22.245025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:22.246780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08cf8a-53cc-4541-b4db-7aa1a34612f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:22.247662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08cf8a-53cc-4541-b4db-7aa1a34612f0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:22.248187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08cf8a-53cc-4541-b4db-7aa1a34612f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:22.248671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08cf8a-53cc-4541-b4db-7aa1a34612f0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:22.249552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08cf8a-53cc-4541-b4db-7aa1a34612f0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:22.255847 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08cf8a-53cc-4541-b4db-7aa1a34612f0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-ff08cf8a-53cc-4541-b4db-7aa1a34612f0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:22.261807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:22.262869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 16:19:22.263762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:22.267435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 362/716] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:22 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:22.274398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08cf8a-53cc-4541-b4db-7aa1a34612f0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:22.301484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:22.375473 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d09a7f-3da8-43e1-aeb4-94879ce16f05 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:22.439826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7496fa-3ec7-4f5d-8559-ef9642649b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:22.440448 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7496fa-3ec7-4f5d-8559-ef9642649b05 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:22.440680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7496fa-3ec7-4f5d-8559-ef9642649b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:22.440932 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7496fa-3ec7-4f5d-8559-ef9642649b05 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:22.441580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7496fa-3ec7-4f5d-8559-ef9642649b05 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:22.442088 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7496fa-3ec7-4f5d-8559-ef9642649b05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:22.443924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7496fa-3ec7-4f5d-8559-ef9642649b05 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:22.456455 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 355/717] 10.222.0.55 () {56 vars in 1124 bytes} [Fri Aug 19 16:19:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:22.468841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08cf8a-53cc-4541-b4db-7aa1a34612f0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:22.509791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f19d2d-9335-4be5-a17a-b368cba82b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:22.510846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f19d2d-9335-4be5-a17a-b368cba82b60 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:22.511347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f19d2d-9335-4be5-a17a-b368cba82b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:22.511857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f19d2d-9335-4be5-a17a-b368cba82b60 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:22.512723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f19d2d-9335-4be5-a17a-b368cba82b60 None admin] RBAC: Adding query filter params (name=network) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:22.513264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f19d2d-9335-4be5-a17a-b368cba82b60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:22.514881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f19d2d-9335-4be5-a17a-b368cba82b60 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:22.520582 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 363/718] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:22.526415 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7496fa-3ec7-4f5d-8559-ef9642649b05 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:22.658538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7496fa-3ec7-4f5d-8559-ef9642649b05 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:22.669176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:22.716526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7750181-3674-4686-876e-c098452835c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:22.717188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7750181-3674-4686-876e-c098452835c8 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:22.717651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7750181-3674-4686-876e-c098452835c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:22.717925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7750181-3674-4686-876e-c098452835c8 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:22.718569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7750181-3674-4686-876e-c098452835c8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:22.718870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7750181-3674-4686-876e-c098452835c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:22.720561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7750181-3674-4686-876e-c098452835c8 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:22.726782 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 356/719] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:22 2022] GET /identity/v3/services?type=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:23.101217 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb53a9-7cf1-49da-86b7-bcc47569b35d None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:23.101558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb53a9-7cf1-49da-86b7-bcc47569b35d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:23.101905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb53a9-7cf1-49da-86b7-bcc47569b35d None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:23.103767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 364/720] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:23.109317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c392c-149d-42de-ac62-1ce0773e378e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:23.109606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c392c-149d-42de-ac62-1ce0773e378e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:23.109882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c392c-149d-42de-ac62-1ce0773e378e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:23.127869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-2b0c392c-149d-42de-ac62-1ce0773e378e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:23.129102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2b0c392c-149d-42de-ac62-1ce0773e378e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:23.216826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 357/721] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 16:19:23.499381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290aecf-cb09-49f1-a59b-07c6f16caff0 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:23.499894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290aecf-cb09-49f1-a59b-07c6f16caff0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:23.499998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290aecf-cb09-49f1-a59b-07c6f16caff0 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:23.501564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 365/722] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:23.507166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aeb64-42e7-437c-bd32-e0df9c0a3ac8 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:23.507575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aeb64-42e7-437c-bd32-e0df9c0a3ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:23.507744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aeb64-42e7-437c-bd32-e0df9c0a3ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:23.537481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-cf9aeb64-42e7-437c-bd32-e0df9c0a3ac8 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:23.537792 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290aecf-cb09-49f1-a59b-07c6f16caff0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:23.538704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-cf9aeb64-42e7-437c-bd32-e0df9c0a3ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:23.644126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 358/723] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:23.654250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:23.687333 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290aecf-cb09-49f1-a59b-07c6f16caff0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:23.722202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97311c51-e0aa-45eb-9b07-55b99bee607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:23.722836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97311c51-e0aa-45eb-9b07-55b99bee607b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:23.723113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97311c51-e0aa-45eb-9b07-55b99bee607b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:23.723385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97311c51-e0aa-45eb-9b07-55b99bee607b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:23.734011 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97311c51-e0aa-45eb-9b07-55b99bee607b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-97311c51-e0aa-45eb-9b07-55b99bee607b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:23.739614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:23.740637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:23.741487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:23.741487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:23.741487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:23.741487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:23.741487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:23.741487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:23.741487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:23.741487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:23.741487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:23.741487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:23.741487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:19:23.741487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:23.744878 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 366/724] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:19:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:23.746976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9aeb64-42e7-437c-bd32-e0df9c0a3ac8 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:23.773917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:23.849703 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9aeb64-42e7-437c-bd32-e0df9c0a3ac8 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:23.885437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d717e7-f12d-46d3-b604-18f947ab7522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:23.886059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d717e7-f12d-46d3-b604-18f947ab7522 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:23.886338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d717e7-f12d-46d3-b604-18f947ab7522 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:23.886610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d717e7-f12d-46d3-b604-18f947ab7522 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:23.887273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d717e7-f12d-46d3-b604-18f947ab7522 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:23.887569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d717e7-f12d-46d3-b604-18f947ab7522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:23.888965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d717e7-f12d-46d3-b604-18f947ab7522 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:23.901061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 359/725] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:23.903614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97311c51-e0aa-45eb-9b07-55b99bee607b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:23.995489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d717e7-f12d-46d3-b604-18f947ab7522 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:24.018109 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97311c51-e0aa-45eb-9b07-55b99bee607b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:24.050566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:24.056540 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb788f-b2f6-4f17-90fe-2475d7efa746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:24.057232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb788f-b2f6-4f17-90fe-2475d7efa746 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:24.057517 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb788f-b2f6-4f17-90fe-2475d7efa746 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:24.057799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb788f-b2f6-4f17-90fe-2475d7efa746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:24.071431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb788f-b2f6-4f17-90fe-2475d7efa746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:24.073014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb788f-b2f6-4f17-90fe-2475d7efa746 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-75eb788f-b2f6-4f17-90fe-2475d7efa746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:24.089538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:24.090535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:24.091432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:24.091432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:24.091432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:24.096124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 367/726] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:24.097414 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb788f-b2f6-4f17-90fe-2475d7efa746 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:24.208544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb788f-b2f6-4f17-90fe-2475d7efa746 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:24.249067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f37dca-f92b-4d7b-8eb0-4e6ddb4243af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:24.249697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f37dca-f92b-4d7b-8eb0-4e6ddb4243af None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:24.250160 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f37dca-f92b-4d7b-8eb0-4e6ddb4243af None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:24.250447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f37dca-f92b-4d7b-8eb0-4e6ddb4243af None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:24.251110 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f37dca-f92b-4d7b-8eb0-4e6ddb4243af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:24.251416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f37dca-f92b-4d7b-8eb0-4e6ddb4243af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:24.252970 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f37dca-f92b-4d7b-8eb0-4e6ddb4243af None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:24.272857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 368/727] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:24.274638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f37dca-f92b-4d7b-8eb0-4e6ddb4243af None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:24.320679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:24.324630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:24.382706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d717e7-f12d-46d3-b604-18f947ab7522 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:24.382940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f37dca-f92b-4d7b-8eb0-4e6ddb4243af None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:24.420575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa43471-7acb-4cef-988f-68d712d6e577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:24.421270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa43471-7acb-4cef-988f-68d712d6e577 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:24.421669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa43471-7acb-4cef-988f-68d712d6e577 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:24.421965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa43471-7acb-4cef-988f-68d712d6e577 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:24.430396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa43471-7acb-4cef-988f-68d712d6e577 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:24.431845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa43471-7acb-4cef-988f-68d712d6e577 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-9aa43471-7acb-4cef-988f-68d712d6e577 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:24.444463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:24.445451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:24.446364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:24.446364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:24.446364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:24.446364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:24.446364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:24.446364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:24.446364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:24.446364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:24.446364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 16:19:24.446364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:24.450228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 369/728] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:19:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:24.451787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa43471-7acb-4cef-988f-68d712d6e577 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:24.515731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4480a-5b31-4995-b03b-a7031c7d15d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:24.516406 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4480a-5b31-4995-b03b-a7031c7d15d1 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:24.516653 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4480a-5b31-4995-b03b-a7031c7d15d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:24.516947 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4480a-5b31-4995-b03b-a7031c7d15d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:24.528658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4480a-5b31-4995-b03b-a7031c7d15d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-79e4480a-5b31-4995-b03b-a7031c7d15d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:24.534035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:24.534953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:24.535770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:24.535770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:24.535770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:24.535770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:24.535770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:24.535770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:24.535770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:24.535770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:24.535770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:24.535770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:24.535770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:24.535770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:24.540713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 360/729] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 540 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:24.542138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fc63e-b339-4769-9336-e997d49771b0 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:24.542426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fc63e-b339-4769-9336-e997d49771b0 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:24.542707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fc63e-b339-4769-9336-e997d49771b0 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:24.544299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 361/730] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:24.545016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fc63e-b339-4769-9336-e997d49771b0 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:24.565402 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa43471-7acb-4cef-988f-68d712d6e577 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:24.568867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:24.616532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255efcd-033a-40ab-abec-373544c11e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:24.617098 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255efcd-033a-40ab-abec-373544c11e25 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:24.617390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255efcd-033a-40ab-abec-373544c11e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:24.618128 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255efcd-033a-40ab-abec-373544c11e25 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:24.618881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255efcd-033a-40ab-abec-373544c11e25 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:24.619322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255efcd-033a-40ab-abec-373544c11e25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:24.621186 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255efcd-033a-40ab-abec-373544c11e25 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:24.649509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 370/731] 10.222.0.55 () {56 vars in 1135 bytes} [Fri Aug 19 16:19:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:24.651118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd968a-733c-4096-8ee3-d43008b55adc None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:24.651604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd968a-733c-4096-8ee3-d43008b55adc None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:24.652091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd968a-733c-4096-8ee3-d43008b55adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:24.673875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-72cd968a-733c-4096-8ee3-d43008b55adc None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:24.675403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-72cd968a-733c-4096-8ee3-d43008b55adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:24.697202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fc63e-b339-4769-9336-e997d49771b0 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:24.735758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3492e4-8fe0-4370-acd0-09a63dd4e244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:24.736346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3492e4-8fe0-4370-acd0-09a63dd4e244 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:24.736628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3492e4-8fe0-4370-acd0-09a63dd4e244 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:24.736936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3492e4-8fe0-4370-acd0-09a63dd4e244 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:24.737600 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3492e4-8fe0-4370-acd0-09a63dd4e244 None admin] RBAC: Adding query filter params (name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:24.737898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3492e4-8fe0-4370-acd0-09a63dd4e244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:24.739299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3492e4-8fe0-4370-acd0-09a63dd4e244 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:24.753875 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 362/732] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:24.753875 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3492e4-8fe0-4370-acd0-09a63dd4e244 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:24.761505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:24.768886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 371/733] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:24.770104 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd968a-733c-4096-8ee3-d43008b55adc None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:24.876491 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3492e4-8fe0-4370-acd0-09a63dd4e244 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:24.912201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:24.913155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4504084-a3eb-426d-80ab-3f8144623563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:24.913755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4504084-a3eb-426d-80ab-3f8144623563 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:24.913871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4504084-a3eb-426d-80ab-3f8144623563 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:24.914083 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4504084-a3eb-426d-80ab-3f8144623563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:24.931734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd968a-733c-4096-8ee3-d43008b55adc None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:24.985219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4504084-a3eb-426d-80ab-3f8144623563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:24.986667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4504084-a3eb-426d-80ab-3f8144623563 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:24.996726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:25.030052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661d973-757d-41d5-8533-49b6c96a6997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:25.030712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661d973-757d-41d5-8533-49b6c96a6997 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.030997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661d973-757d-41d5-8533-49b6c96a6997 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.031273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661d973-757d-41d5-8533-49b6c96a6997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.057908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661d973-757d-41d5-8533-49b6c96a6997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:25.059404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661d973-757d-41d5-8533-49b6c96a6997 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-8661d973-757d-41d5-8533-49b6c96a6997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:25.090950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:25.091993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:25.092876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:25.092876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:25.092876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:25.092876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 372/735] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:25.094563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 363/735] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:19:25.097714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661d973-757d-41d5-8533-49b6c96a6997 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:25.101740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4504084-a3eb-426d-80ab-3f8144623563 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:25.205502 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4504084-a3eb-426d-80ab-3f8144623563 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:25.213434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661d973-757d-41d5-8533-49b6c96a6997 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:25.215688 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:25.241858 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77659b-f9a1-4549-b926-830f6d339eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:25.242464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77659b-f9a1-4549-b926-830f6d339eaa None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.242740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77659b-f9a1-4549-b926-830f6d339eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.243013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77659b-f9a1-4549-b926-830f6d339eaa None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.243650 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77659b-f9a1-4549-b926-830f6d339eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:25.243950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77659b-f9a1-4549-b926-830f6d339eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:25.245343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77659b-f9a1-4549-b926-830f6d339eaa None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:25.248945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0881bc-ee3e-4c22-a0f1-53b45161fce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:25.249574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0881bc-ee3e-4c22-a0f1-53b45161fce9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.249852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0881bc-ee3e-4c22-a0f1-53b45161fce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.250124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0881bc-ee3e-4c22-a0f1-53b45161fce9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.256970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 364/736] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:25.258145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe103e-df3b-44e2-8534-9571828c88aa None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.258427 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe103e-df3b-44e2-8534-9571828c88aa None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.258699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe103e-df3b-44e2-8534-9571828c88aa None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.260030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 365/737] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:25.260543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe103e-df3b-44e2-8534-9571828c88aa None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:25.265732 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0881bc-ee3e-4c22-a0f1-53b45161fce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:25.267181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0881bc-ee3e-4c22-a0f1-53b45161fce9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:25.269785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-bb0881bc-ee3e-4c22-a0f1-53b45161fce9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:25.282026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:25.287789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:25.288783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:25.288783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:25.288783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:25.288783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 373/738] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:25.288783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2219a-08e6-4621-8f46-1a8df5233d55 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.288783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2219a-08e6-4621-8f46-1a8df5233d55 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.288783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2219a-08e6-4621-8f46-1a8df5233d55 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.288783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 374/739] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:25.288783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2219a-08e6-4621-8f46-1a8df5233d55 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:25.366103 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe103e-df3b-44e2-8534-9571828c88aa None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:25.394300 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2219a-08e6-4621-8f46-1a8df5233d55 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:25.399694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527db71-478f-4389-bc74-fcab13245ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:25.400320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527db71-478f-4389-bc74-fcab13245ebe None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.400599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527db71-478f-4389-bc74-fcab13245ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.400900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527db71-478f-4389-bc74-fcab13245ebe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.411564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7527db71-478f-4389-bc74-fcab13245ebe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-7527db71-478f-4389-bc74-fcab13245ebe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:25.416212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:25.417230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:25.418053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 366/740] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:19:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:25.418763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56816e-a406-4d27-be4c-5b12fd42e68e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.418763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56816e-a406-4d27-be4c-5b12fd42e68e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.419149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56816e-a406-4d27-be4c-5b12fd42e68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.430634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d8bdc-8214-4ee8-9e37-7e34e3865cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:25.431261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d8bdc-8214-4ee8-9e37-7e34e3865cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.431539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d8bdc-8214-4ee8-9e37-7e34e3865cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.431811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d8bdc-8214-4ee8-9e37-7e34e3865cb0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.437564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ba56816e-a406-4d27-be4c-5b12fd42e68e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:25.438752 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ba56816e-a406-4d27-be4c-5b12fd42e68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:25.439264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d8bdc-8214-4ee8-9e37-7e34e3865cb0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:25.440689 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d8bdc-8214-4ee8-9e37-7e34e3865cb0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-3a9d8bdc-8214-4ee8-9e37-7e34e3865cb0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:25.452161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:25.453343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:25.454275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:25.454275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:25.454275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:25.454275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:25.454275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:25.454275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:25.454275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:25.454275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:25.454275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 16:19:25.454275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:25.454275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 375/741] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Aug 19 16:19:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:25.454275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d8bdc-8214-4ee8-9e37-7e34e3865cb0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:25.465246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:25.508661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:25.511245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 367/742] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:25.512563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efeb67-c27b-411c-ba58-82a4d2bd5afa None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.512875 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efeb67-c27b-411c-ba58-82a4d2bd5afa None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.513157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efeb67-c27b-411c-ba58-82a4d2bd5afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.539616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-43efeb67-c27b-411c-ba58-82a4d2bd5afa None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:25.540845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-43efeb67-c27b-411c-ba58-82a4d2bd5afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:25.554154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d8bdc-8214-4ee8-9e37-7e34e3865cb0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:25.588689 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e611b-9e1c-4bb3-a5b9-fc16f5c87f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:25.589309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e611b-9e1c-4bb3-a5b9-fc16f5c87f32 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.589590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e611b-9e1c-4bb3-a5b9-fc16f5c87f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.589863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e611b-9e1c-4bb3-a5b9-fc16f5c87f32 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.590512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e611b-9e1c-4bb3-a5b9-fc16f5c87f32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:25.590815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e611b-9e1c-4bb3-a5b9-fc16f5c87f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:25.592206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e611b-9e1c-4bb3-a5b9-fc16f5c87f32 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:25.604561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 376/743] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:25.605389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e611b-9e1c-4bb3-a5b9-fc16f5c87f32 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:25.616282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 368/744] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 16:19:25.617533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efeb67-c27b-411c-ba58-82a4d2bd5afa None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:25.733746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e611b-9e1c-4bb3-a5b9-fc16f5c87f32 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:25.742904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:25.755403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:25.759528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efeb67-c27b-411c-ba58-82a4d2bd5afa None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:25.775785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734b46d-fe4a-4318-9951-89f7b0db1158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:25.777142 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734b46d-fe4a-4318-9951-89f7b0db1158 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.777426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734b46d-fe4a-4318-9951-89f7b0db1158 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.777702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734b46d-fe4a-4318-9951-89f7b0db1158 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.778368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734b46d-fe4a-4318-9951-89f7b0db1158 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:25.778671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734b46d-fe4a-4318-9951-89f7b0db1158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:25.780056 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734b46d-fe4a-4318-9951-89f7b0db1158 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:25.789680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 377/745] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:25.794001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734b46d-fe4a-4318-9951-89f7b0db1158 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:25.799445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d63826-eed1-427c-84ec-a9b25e8264b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:25.800054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d63826-eed1-427c-84ec-a9b25e8264b2 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.800332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d63826-eed1-427c-84ec-a9b25e8264b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.800605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d63826-eed1-427c-84ec-a9b25e8264b2 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.801373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d63826-eed1-427c-84ec-a9b25e8264b2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:25.802222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d63826-eed1-427c-84ec-a9b25e8264b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:25.803731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d63826-eed1-427c-84ec-a9b25e8264b2 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:25.822183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 369/746] 10.222.0.55 () {56 vars in 1149 bytes} [Fri Aug 19 16:19:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:25.822817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d63826-eed1-427c-84ec-a9b25e8264b2 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:25.904726 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734b46d-fe4a-4318-9951-89f7b0db1158 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:25.924096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d63826-eed1-427c-84ec-a9b25e8264b2 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:25.940258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0d290-fd69-4b55-a6d1-22138e7ab485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:25.940921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d290-fd69-4b55-a6d1-22138e7ab485 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.941200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d290-fd69-4b55-a6d1-22138e7ab485 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.941472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d290-fd69-4b55-a6d1-22138e7ab485 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.948099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0d290-fd69-4b55-a6d1-22138e7ab485 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:25.949700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0d290-fd69-4b55-a6d1-22138e7ab485 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:25.957691 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f78ac-b353-4628-b3a7-f593011c7280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:25.958890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f78ac-b353-4628-b3a7-f593011c7280 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:25.959166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f78ac-b353-4628-b3a7-f593011c7280 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:25.959433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f78ac-b353-4628-b3a7-f593011c7280 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:25.960062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f78ac-b353-4628-b3a7-f593011c7280 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-49d0d290-fd69-4b55-a6d1-22138e7ab485 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:25.961126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:25.962136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:25.963021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:25.963021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:25.963021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:25.963021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:25.963021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:25.963021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:25.963021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:25.963021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:25.963425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f78ac-b353-4628-b3a7-f593011c7280 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:25.963676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 378/747] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:25.963676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0d290-fd69-4b55-a6d1-22138e7ab485 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:25.979499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 370/748] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:19:25 2022] POST /identity/v3/services => generated 240 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:25.980387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f78ac-b353-4628-b3a7-f593011c7280 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:25.994426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:26.009454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:26.081114 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0d290-fd69-4b55-a6d1-22138e7ab485 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:26.119001 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f78ac-b353-4628-b3a7-f593011c7280 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:26.122250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681b0c8-9315-4eaf-9f44-262ed671af09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:26.122410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681b0c8-9315-4eaf-9f44-262ed671af09 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:26.122576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681b0c8-9315-4eaf-9f44-262ed671af09 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:26.122846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681b0c8-9315-4eaf-9f44-262ed671af09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:26.135374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681b0c8-9315-4eaf-9f44-262ed671af09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:26.137486 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681b0c8-9315-4eaf-9f44-262ed671af09 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a681b0c8-9315-4eaf-9f44-262ed671af09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:26.152891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:26.153869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:26.154737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:26.154737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:26.154737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:26.154737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 379/749] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:26.154737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681b0c8-9315-4eaf-9f44-262ed671af09 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:26.163888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29569a85-556d-47bb-a5fc-57be3465fe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:26.164909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29569a85-556d-47bb-a5fc-57be3465fe9a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:26.165398 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29569a85-556d-47bb-a5fc-57be3465fe9a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:26.165736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29569a85-556d-47bb-a5fc-57be3465fe9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:26.239769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29569a85-556d-47bb-a5fc-57be3465fe9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:26.241321 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29569a85-556d-47bb-a5fc-57be3465fe9a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:26.243330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:26.253216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:26.264410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681b0c8-9315-4eaf-9f44-262ed671af09 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:26.301152 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4b231-2c42-4e03-8c3d-b2d9411f1f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:26.301931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4b231-2c42-4e03-8c3d-b2d9411f1f24 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:26.302212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4b231-2c42-4e03-8c3d-b2d9411f1f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:26.302572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4b231-2c42-4e03-8c3d-b2d9411f1f24 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:26.303212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4b231-2c42-4e03-8c3d-b2d9411f1f24 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:26.304589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4b231-2c42-4e03-8c3d-b2d9411f1f24 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:26.321304 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 380/750] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:19:26 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:26.321981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4b231-2c42-4e03-8c3d-b2d9411f1f24 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:26.469647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4b231-2c42-4e03-8c3d-b2d9411f1f24 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:26.483191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 371/751] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:25 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 16:19:26.484351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29569a85-556d-47bb-a5fc-57be3465fe9a None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:26.517716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc25f90-aabb-4cc3-bda3-d1e5f10945b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:26.518666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc25f90-aabb-4cc3-bda3-d1e5f10945b7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:26.518949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc25f90-aabb-4cc3-bda3-d1e5f10945b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:26.519225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc25f90-aabb-4cc3-bda3-d1e5f10945b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:26.519928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc25f90-aabb-4cc3-bda3-d1e5f10945b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:26.520832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc25f90-aabb-4cc3-bda3-d1e5f10945b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:26.522366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc25f90-aabb-4cc3-bda3-d1e5f10945b7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:26.537410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 381/752] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:26.538103 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc25f90-aabb-4cc3-bda3-d1e5f10945b7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:26.547052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:26.600634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29569a85-556d-47bb-a5fc-57be3465fe9a None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:26.609894 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:26.643907 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe1ebe-293f-4c74-a6d2-3034e3e0d3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:26.644630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe1ebe-293f-4c74-a6d2-3034e3e0d3cc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:26.644952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe1ebe-293f-4c74-a6d2-3034e3e0d3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:26.645258 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe1ebe-293f-4c74-a6d2-3034e3e0d3cc None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:26.646610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe1ebe-293f-4c74-a6d2-3034e3e0d3cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:26.646915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe1ebe-293f-4c74-a6d2-3034e3e0d3cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:26.648910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe1ebe-293f-4c74-a6d2-3034e3e0d3cc None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:26.665492 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 372/753] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:26.667024 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe1ebe-293f-4c74-a6d2-3034e3e0d3cc None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:26.668994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc25f90-aabb-4cc3-bda3-d1e5f10945b7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:26.720478 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687c874-242d-4b8c-9050-5fa941193af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:26.721153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687c874-242d-4b8c-9050-5fa941193af0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:26.721434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687c874-242d-4b8c-9050-5fa941193af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:26.721711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687c874-242d-4b8c-9050-5fa941193af0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:26.736522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687c874-242d-4b8c-9050-5fa941193af0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:26.736522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687c874-242d-4b8c-9050-5fa941193af0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-e687c874-242d-4b8c-9050-5fa941193af0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:26.759867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:26.760982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:26.761850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:26.761850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:26.761850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:26.762015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 382/754] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:26.763302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687c874-242d-4b8c-9050-5fa941193af0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:26.789616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe1ebe-293f-4c74-a6d2-3034e3e0d3cc None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:26.832220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95272bbb-c174-42e0-924f-702f0e9dea28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:26.833066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95272bbb-c174-42e0-924f-702f0e9dea28 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:26.833440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95272bbb-c174-42e0-924f-702f0e9dea28 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:26.833816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95272bbb-c174-42e0-924f-702f0e9dea28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:26.835156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95272bbb-c174-42e0-924f-702f0e9dea28 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=c755f9538e984bc3877112896595e158) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:26.835539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95272bbb-c174-42e0-924f-702f0e9dea28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:26.837408 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95272bbb-c174-42e0-924f-702f0e9dea28 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:26.844576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 373/755] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:26 2022] GET /identity/v3/role_assignments?user.id=c755f9538e984bc3877112896595e158&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:26.845492 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:26.846452 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95272bbb-c174-42e0-924f-702f0e9dea28 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:26.881923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:26.936153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687c874-242d-4b8c-9050-5fa941193af0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:26.995795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7da52-4ee2-4848-8cff-6574517c4b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:26.997733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7da52-4ee2-4848-8cff-6574517c4b92 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:26.998667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7da52-4ee2-4848-8cff-6574517c4b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:26.999741 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7da52-4ee2-4848-8cff-6574517c4b92 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:27.002559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95272bbb-c174-42e0-924f-702f0e9dea28 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:27.013054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7da52-4ee2-4848-8cff-6574517c4b92 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:27.017867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7da52-4ee2-4848-8cff-6574517c4b92 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4bf7da52-4ee2-4848-8cff-6574517c4b92 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:27.043883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:27.044841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:27.045767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:27.045767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:27.045767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:27.045767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:27.045767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:27.045767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:27.045767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:27.045767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:27.045767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:19:27.045767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:27.045767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 383/756] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Aug 19 16:19:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:27.045767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7da52-4ee2-4848-8cff-6574517c4b92 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:27.059878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474bdf2-1b9c-4871-be3d-ac6f62ca27ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:27.060903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474bdf2-1b9c-4871-be3d-ac6f62ca27ca None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:27.061381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474bdf2-1b9c-4871-be3d-ac6f62ca27ca None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:27.062346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474bdf2-1b9c-4871-be3d-ac6f62ca27ca None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:27.063192 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474bdf2-1b9c-4871-be3d-ac6f62ca27ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:27.064055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474bdf2-1b9c-4871-be3d-ac6f62ca27ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:27.065761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474bdf2-1b9c-4871-be3d-ac6f62ca27ca None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:27.081419 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 374/757] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:27.090309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474bdf2-1b9c-4871-be3d-ac6f62ca27ca None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:27.253698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:27.255302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7da52-4ee2-4848-8cff-6574517c4b92 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:27.278866 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474bdf2-1b9c-4871-be3d-ac6f62ca27ca None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:27.290319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:27.315881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3764e-2f76-4be3-972a-1206419958da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:27.316416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3764e-2f76-4be3-972a-1206419958da None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:27.316696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3764e-2f76-4be3-972a-1206419958da None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:27.316998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3764e-2f76-4be3-972a-1206419958da None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:27.317858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3764e-2f76-4be3-972a-1206419958da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:27.318228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3764e-2f76-4be3-972a-1206419958da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:27.319663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3764e-2f76-4be3-972a-1206419958da None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:27.333271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8eaab-4497-4415-8774-b2d1e441956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:27.333916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8eaab-4497-4415-8774-b2d1e441956f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:27.334201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8eaab-4497-4415-8774-b2d1e441956f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:27.334480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8eaab-4497-4415-8774-b2d1e441956f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:27.341384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8eaab-4497-4415-8774-b2d1e441956f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:27.342846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8eaab-4497-4415-8774-b2d1e441956f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:27.345972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 384/758] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Aug 19 16:19:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:27.347206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3764e-2f76-4be3-972a-1206419958da None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-e3b8eaab-4497-4415-8774-b2d1e441956f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:27.355402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:27.356474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:27.357474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:27.357474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:27.357474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:27.357474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:27.357474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:27.357474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:27.357474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:27.357474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:27.359681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 375/759] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:27.369920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8eaab-4497-4415-8774-b2d1e441956f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:27.560462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3764e-2f76-4be3-972a-1206419958da None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:27.564433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8eaab-4497-4415-8774-b2d1e441956f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:27.623789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be377ef-1321-4172-a267-86bc66ccb5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:27.624429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be377ef-1321-4172-a267-86bc66ccb5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:27.624639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be377ef-1321-4172-a267-86bc66ccb5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:27.624879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be377ef-1321-4172-a267-86bc66ccb5a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:27.625569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be377ef-1321-4172-a267-86bc66ccb5a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:27.625873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be377ef-1321-4172-a267-86bc66ccb5a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:27.627316 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be377ef-1321-4172-a267-86bc66ccb5a6 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:27.649361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f4a3a-74ac-4dc6-b174-3b5f12d9fe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:27.650008 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f4a3a-74ac-4dc6-b174-3b5f12d9fe2c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:27.650288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f4a3a-74ac-4dc6-b174-3b5f12d9fe2c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:27.650560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f4a3a-74ac-4dc6-b174-3b5f12d9fe2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:27.656304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 376/760] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:27.663121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be377ef-1321-4172-a267-86bc66ccb5a6 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:27.675535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f4a3a-74ac-4dc6-b174-3b5f12d9fe2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:27.676436 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:27.677508 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f4a3a-74ac-4dc6-b174-3b5f12d9fe2c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-d30f4a3a-74ac-4dc6-b174-3b5f12d9fe2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:27.703812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:27.704797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:27.705676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:27.705676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:27.705676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:27.712291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 385/761] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:27.713161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f4a3a-74ac-4dc6-b174-3b5f12d9fe2c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:27.745101 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:27.829528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f4a3a-74ac-4dc6-b174-3b5f12d9fe2c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:27.884265 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30209227-ad0c-4837-9de7-b474b64ac026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:27.885264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30209227-ad0c-4837-9de7-b474b64ac026 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:27.885772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30209227-ad0c-4837-9de7-b474b64ac026 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:27.886239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30209227-ad0c-4837-9de7-b474b64ac026 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:27.887091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30209227-ad0c-4837-9de7-b474b64ac026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:27.887586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30209227-ad0c-4837-9de7-b474b64ac026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:27.889182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30209227-ad0c-4837-9de7-b474b64ac026 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:27.901187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be377ef-1321-4172-a267-86bc66ccb5a6 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:27.912234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 386/762] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:27.922200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30209227-ad0c-4837-9de7-b474b64ac026 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:27.933082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733d4e6-1531-4f38-96f5-553948e8f0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:27.940865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733d4e6-1531-4f38-96f5-553948e8f0ad None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:27.940865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733d4e6-1531-4f38-96f5-553948e8f0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:27.940865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733d4e6-1531-4f38-96f5-553948e8f0ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:27.940865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733d4e6-1531-4f38-96f5-553948e8f0ad None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=88694a50b9f04487a521ba6df110d976) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:27.940865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733d4e6-1531-4f38-96f5-553948e8f0ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:27.940865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733d4e6-1531-4f38-96f5-553948e8f0ad None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:27.943620 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 377/763] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:27 2022] GET /identity/v3/role_assignments?user.id=88694a50b9f04487a521ba6df110d976&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:28.072382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:28.090980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30209227-ad0c-4837-9de7-b474b64ac026 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:28.141811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cec206-0b2a-4b35-b10b-cd299ef0dee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:28.142374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cec206-0b2a-4b35-b10b-cd299ef0dee5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:28.142655 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cec206-0b2a-4b35-b10b-cd299ef0dee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:28.142929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cec206-0b2a-4b35-b10b-cd299ef0dee5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:28.154684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cec206-0b2a-4b35-b10b-cd299ef0dee5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:28.156140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cec206-0b2a-4b35-b10b-cd299ef0dee5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-06cec206-0b2a-4b35-b10b-cd299ef0dee5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:28.169819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:28.170928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:28.171904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:28.171904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:28.171904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:28.171904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:28.171904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:28.171904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:28.171904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:28.171904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:28.171904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 387/764] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:28.177539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733d4e6-1531-4f38-96f5-553948e8f0ad None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:28.216933 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e641d-6810-4334-9ce7-ddbdfab337e3 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:28.217260 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e641d-6810-4334-9ce7-ddbdfab337e3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:28.217491 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e641d-6810-4334-9ce7-ddbdfab337e3 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:28.221723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 388/765] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:28.241585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221c69e-1ffb-4954-aaa3-cca396180ba5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:28.241862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221c69e-1ffb-4954-aaa3-cca396180ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:28.242130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221c69e-1ffb-4954-aaa3-cca396180ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:28.267243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8221c69e-1ffb-4954-aaa3-cca396180ba5 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:28.268451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8221c69e-1ffb-4954-aaa3-cca396180ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:28.304956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:28.358305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733d4e6-1531-4f38-96f5-553948e8f0ad None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:28.403558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a0431-2988-4673-9cfb-0fd7e187f1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:28.404300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a0431-2988-4673-9cfb-0fd7e187f1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:28.404582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a0431-2988-4673-9cfb-0fd7e187f1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:28.404936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a0431-2988-4673-9cfb-0fd7e187f1a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:28.405691 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a0431-2988-4673-9cfb-0fd7e187f1a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:28.406249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a0431-2988-4673-9cfb-0fd7e187f1a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:28.406358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 389/766] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:28.407428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a0431-2988-4673-9cfb-0fd7e187f1a9 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:28.420580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 378/767] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:28.426188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221c69e-1ffb-4954-aaa3-cca396180ba5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:28.541227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:28.543278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4324cd-0d08-4c06-901b-2b432d7c7332 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:28.543564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4324cd-0d08-4c06-901b-2b432d7c7332 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:28.543838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4324cd-0d08-4c06-901b-2b432d7c7332 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:28.545199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 379/768] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:19:28.553493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80d673-505b-4903-9cc9-79148c4b775e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:28.553764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80d673-505b-4903-9cc9-79148c4b775e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:28.554064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80d673-505b-4903-9cc9-79148c4b775e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:28.578266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-5f80d673-505b-4903-9cc9-79148c4b775e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:28.579472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5f80d673-505b-4903-9cc9-79148c4b775e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:28.593037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221c69e-1ffb-4954-aaa3-cca396180ba5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:28.637162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43565bb6-41b1-4dad-834a-1e3be1cf2bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:28.637841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43565bb6-41b1-4dad-834a-1e3be1cf2bd3 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:28.638127 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43565bb6-41b1-4dad-834a-1e3be1cf2bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:28.638405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43565bb6-41b1-4dad-834a-1e3be1cf2bd3 None admin] PATH_INFO: `/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/4739e80619a54b178b9aacec6a42a7f0/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:28.640383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43565bb6-41b1-4dad-834a-1e3be1cf2bd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d143cf444f354ec0bf2089dae3d94255, user_id=4739e80619a54b178b9aacec6a42a7f0, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:28.642092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43565bb6-41b1-4dad-834a-1e3be1cf2bd3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:28.658927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 390/769] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:19:28 2022] PUT /identity/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/4739e80619a54b178b9aacec6a42a7f0/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:19:28.715221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 380/770] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:28.764670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43565bb6-41b1-4dad-834a-1e3be1cf2bd3 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:28.915012 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43565bb6-41b1-4dad-834a-1e3be1cf2bd3 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:28.970801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510974a-7ee9-46e4-8e9c-8fff9b38b459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:28.971506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510974a-7ee9-46e4-8e9c-8fff9b38b459 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:28.971808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510974a-7ee9-46e4-8e9c-8fff9b38b459 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:28.972065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510974a-7ee9-46e4-8e9c-8fff9b38b459 None admin] PATH_INFO: `/v3/services/network` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:28.972659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510974a-7ee9-46e4-8e9c-8fff9b38b459 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:28.975021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510974a-7ee9-46e4-8e9c-8fff9b38b459 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-1510974a-7ee9-46e4-8e9c-8fff9b38b459 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:28.984401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:28.985423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:28.986248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 391/771] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:28 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:28.991314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80d673-505b-4903-9cc9-79148c4b775e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:29.010221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8242430-d8de-4bf9-bcf7-c8e14c28228d None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:29.010852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8242430-d8de-4bf9-bcf7-c8e14c28228d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:29.011344 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8242430-d8de-4bf9-bcf7-c8e14c28228d None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:29.013242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 392/772] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:19:29.017442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8242430-d8de-4bf9-bcf7-c8e14c28228d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:29.020933 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:29.067548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:29.135405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80d673-505b-4903-9cc9-79148c4b775e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:29.139389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8242430-d8de-4bf9-bcf7-c8e14c28228d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:29.176110 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a37e83-5bb7-474e-ad75-060281a47c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:29.177345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a37e83-5bb7-474e-ad75-060281a47c6a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:29.177749 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a37e83-5bb7-474e-ad75-060281a47c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:29.178025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a37e83-5bb7-474e-ad75-060281a47c6a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:29.178613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a37e83-5bb7-474e-ad75-060281a47c6a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:29.179996 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a37e83-5bb7-474e-ad75-060281a47c6a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:29.181488 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cb404-d1b1-4d85-a34c-5b9371798c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:29.182400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb404-d1b1-4d85-a34c-5b9371798c60 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:29.183088 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb404-d1b1-4d85-a34c-5b9371798c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:29.183374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb404-d1b1-4d85-a34c-5b9371798c60 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:29.184311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cb404-d1b1-4d85-a34c-5b9371798c60 None admin] RBAC: Adding query filter params (name=network) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:29.184623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cb404-d1b1-4d85-a34c-5b9371798c60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:29.186022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cb404-d1b1-4d85-a34c-5b9371798c60 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-48a37e83-5bb7-474e-ad75-060281a47c6a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:29.190350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:29.191623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 16:19:29.193257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:29.194009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 381/773] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:29.194009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24514101-d88f-47fd-a102-c07afc819c06 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:29.194009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24514101-d88f-47fd-a102-c07afc819c06 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:29.194009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24514101-d88f-47fd-a102-c07afc819c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:29.196536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 393/774] 10.222.0.55 () {56 vars in 1129 bytes} [Fri Aug 19 16:19:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:29.197126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a37e83-5bb7-474e-ad75-060281a47c6a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:29.224441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-24514101-d88f-47fd-a102-c07afc819c06 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:29.225654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-24514101-d88f-47fd-a102-c07afc819c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:29.275587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:29.295756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 382/775] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:29.296613 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24514101-d88f-47fd-a102-c07afc819c06 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:29.316229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a37e83-5bb7-474e-ad75-060281a47c6a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:29.353876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:29.359092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283fff2-7815-4ebe-a6df-1404ede01cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:29.359847 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283fff2-7815-4ebe-a6df-1404ede01cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:29.360225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283fff2-7815-4ebe-a6df-1404ede01cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:29.360603 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283fff2-7815-4ebe-a6df-1404ede01cf3 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:29.361339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283fff2-7815-4ebe-a6df-1404ede01cf3 None admin] RBAC: Adding query filter params (type=network) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:29.361769 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283fff2-7815-4ebe-a6df-1404ede01cf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:29.363220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283fff2-7815-4ebe-a6df-1404ede01cf3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:29.377824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 394/776] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:29.385605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283fff2-7815-4ebe-a6df-1404ede01cf3 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:29.407985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24514101-d88f-47fd-a102-c07afc819c06 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:29.444110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36945c25-df7d-4fc8-801e-61431133589e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:29.444967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36945c25-df7d-4fc8-801e-61431133589e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:29.445409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36945c25-df7d-4fc8-801e-61431133589e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:29.445813 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36945c25-df7d-4fc8-801e-61431133589e None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:29.446593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36945c25-df7d-4fc8-801e-61431133589e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:29.447029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36945c25-df7d-4fc8-801e-61431133589e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:29.448519 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36945c25-df7d-4fc8-801e-61431133589e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:29.453621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 383/777] 10.222.0.55 () {56 vars in 1124 bytes} [Fri Aug 19 16:19:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:29.463571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36945c25-df7d-4fc8-801e-61431133589e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:29.589859 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283fff2-7815-4ebe-a6df-1404ede01cf3 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:29.646577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:29.657759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe83a1-9445-47f2-b440-57bed9ac9589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:29.658350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe83a1-9445-47f2-b440-57bed9ac9589 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:29.658634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe83a1-9445-47f2-b440-57bed9ac9589 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:29.658909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe83a1-9445-47f2-b440-57bed9ac9589 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:29.659629 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe83a1-9445-47f2-b440-57bed9ac9589 None admin] RBAC: Adding query filter params (interface=public, service_id=4422db1f395542e1ab74ce1b93476485, region_id=RegionOne) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:29.660062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe83a1-9445-47f2-b440-57bed9ac9589 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:29.662932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe83a1-9445-47f2-b440-57bed9ac9589 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:29.665114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36945c25-df7d-4fc8-801e-61431133589e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:29.673176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 395/778] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Aug 19 16:19:29 2022] GET /identity/v3/endpoints?service_id=4422db1f395542e1ab74ce1b93476485&interface=public®ion_id=RegionOne => generated 194 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:29.674018 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe83a1-9445-47f2-b440-57bed9ac9589 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:29.691259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:29.711227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab58808-5bb8-4918-be96-30e5cd287945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:29.711771 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab58808-5bb8-4918-be96-30e5cd287945 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:29.712168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab58808-5bb8-4918-be96-30e5cd287945 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:29.712421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab58808-5bb8-4918-be96-30e5cd287945 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:29.713486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab58808-5bb8-4918-be96-30e5cd287945 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:29.713727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab58808-5bb8-4918-be96-30e5cd287945 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:29.715106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab58808-5bb8-4918-be96-30e5cd287945 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:29.724779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 384/779] 10.222.0.55 () {56 vars in 1124 bytes} [Fri Aug 19 16:19:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:29.724779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab58808-5bb8-4918-be96-30e5cd287945 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:29.802555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe83a1-9445-47f2-b440-57bed9ac9589 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:29.828465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab58808-5bb8-4918-be96-30e5cd287945 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:29.840976 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fc79c-64a1-4e01-ac46-75a9728c0c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:29.841617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fc79c-64a1-4e01-ac46-75a9728c0c5c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:29.841902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fc79c-64a1-4e01-ac46-75a9728c0c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:29.842181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fc79c-64a1-4e01-ac46-75a9728c0c5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:29.854111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fc79c-64a1-4e01-ac46-75a9728c0c5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-964fc79c-64a1-4e01-ac46-75a9728c0c5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:29.859969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:29.860893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:29.861736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 396/780] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:19:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:29.862405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fc79c-64a1-4e01-ac46-75a9728c0c5c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:29.868679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd9248-fac9-41f3-b0cf-e3f043b5aca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:29.869593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd9248-fac9-41f3-b0cf-e3f043b5aca7 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:29.870086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd9248-fac9-41f3-b0cf-e3f043b5aca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:29.870549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd9248-fac9-41f3-b0cf-e3f043b5aca7 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:29.871340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd9248-fac9-41f3-b0cf-e3f043b5aca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:29.872912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd9248-fac9-41f3-b0cf-e3f043b5aca7 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:29.884909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 385/781] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Aug 19 16:19:29 2022] GET /identity/v3/services => generated 772 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:29.886229 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd9248-fac9-41f3-b0cf-e3f043b5aca7 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:29.928028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:29.972000 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:29.981762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fc79c-64a1-4e01-ac46-75a9728c0c5c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:30.021672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4fe2f-4956-43f6-a3d5-761f5023ec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:30.022909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4fe2f-4956-43f6-a3d5-761f5023ec5a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:30.023437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4fe2f-4956-43f6-a3d5-761f5023ec5a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:30.023919 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4fe2f-4956-43f6-a3d5-761f5023ec5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:30.025073 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4fe2f-4956-43f6-a3d5-761f5023ec5a None admin] RBAC: Adding query filter params (interface=public, service_id=ba6245ecfda4458bb97f1a6fdb5640e5, region_id=RegionOne) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:30.025605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4fe2f-4956-43f6-a3d5-761f5023ec5a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:30.027305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4fe2f-4956-43f6-a3d5-761f5023ec5a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:30.034068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 397/782] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Aug 19 16:19:29 2022] GET /identity/v3/endpoints?service_id=ba6245ecfda4458bb97f1a6fdb5640e5&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:30.042944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4fe2f-4956-43f6-a3d5-761f5023ec5a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:30.079458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd9248-fac9-41f3-b0cf-e3f043b5aca7 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:30.127240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f04e34-6e1a-40f5-a84e-675514adb80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:30.128148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f04e34-6e1a-40f5-a84e-675514adb80f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:30.128425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f04e34-6e1a-40f5-a84e-675514adb80f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:30.128699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f04e34-6e1a-40f5-a84e-675514adb80f None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:30.129722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f04e34-6e1a-40f5-a84e-675514adb80f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:30.130487 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f04e34-6e1a-40f5-a84e-675514adb80f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:30.132126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f04e34-6e1a-40f5-a84e-675514adb80f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:30.143554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 386/783] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:30.145061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fec3ca-347c-477b-853d-3bf4bae43fbc None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:30.145352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fec3ca-347c-477b-853d-3bf4bae43fbc None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:30.145626 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fec3ca-347c-477b-853d-3bf4bae43fbc None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:30.146938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 387/784] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:30.151866 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fec3ca-347c-477b-853d-3bf4bae43fbc None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:30.183630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4fe2f-4956-43f6-a3d5-761f5023ec5a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:30.202118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:30.227925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a853a-8378-4bc6-80b7-b1e5d353e6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:30.228792 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a853a-8378-4bc6-80b7-b1e5d353e6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:30.229080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a853a-8378-4bc6-80b7-b1e5d353e6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:30.229381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a853a-8378-4bc6-80b7-b1e5d353e6c5 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:30.230594 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a853a-8378-4bc6-80b7-b1e5d353e6c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:30.232247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a853a-8378-4bc6-80b7-b1e5d353e6c5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:30.240535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 398/785] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Aug 19 16:19:30 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:30.241835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9e529-6614-4940-9acd-97b0fceed972 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:30.242127 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9e529-6614-4940-9acd-97b0fceed972 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:30.242408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9e529-6614-4940-9acd-97b0fceed972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:30.263810 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-64c9e529-6614-4940-9acd-97b0fceed972 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:30.265186 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-64c9e529-6614-4940-9acd-97b0fceed972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:30.326814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fec3ca-347c-477b-853d-3bf4bae43fbc None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:30.360780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:30.371458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba108-a288-4e91-bbd0-e6eeac5ee84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:30.372106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba108-a288-4e91-bbd0-e6eeac5ee84c None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:30.374023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba108-a288-4e91-bbd0-e6eeac5ee84c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:30.374319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba108-a288-4e91-bbd0-e6eeac5ee84c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:30.401942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bba108-a288-4e91-bbd0-e6eeac5ee84c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:30.403411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bba108-a288-4e91-bbd0-e6eeac5ee84c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:30.420015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 399/786] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-f3bba108-a288-4e91-bbd0-e6eeac5ee84c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:30.431517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:30.432485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:30.433386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:30.433386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:30.433386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:30.433386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 388/787] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:30.439443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9e529-6614-4940-9acd-97b0fceed972 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:30.558218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9e529-6614-4940-9acd-97b0fceed972 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:30.576990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:30.601106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b7ede-5ea0-4292-9650-442c67701bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:30.601727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b7ede-5ea0-4292-9650-442c67701bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:30.602011 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b7ede-5ea0-4292-9650-442c67701bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:30.602544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b7ede-5ea0-4292-9650-442c67701bf8 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:30.603197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b7ede-5ea0-4292-9650-442c67701bf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:30.603498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b7ede-5ea0-4292-9650-442c67701bf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:30.605017 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b7ede-5ea0-4292-9650-442c67701bf8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:30.617654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 400/788] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:30.623131 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba108-a288-4e91-bbd0-e6eeac5ee84c None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:30.732676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b7ede-5ea0-4292-9650-442c67701bf8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:30.766913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba108-a288-4e91-bbd0-e6eeac5ee84c None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:30.807644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb451de9-ea85-428d-abca-cafdf2b0cf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:30.808332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb451de9-ea85-428d-abca-cafdf2b0cf1d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:30.808555 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb451de9-ea85-428d-abca-cafdf2b0cf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:30.808850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb451de9-ea85-428d-abca-cafdf2b0cf1d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:30.821928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb451de9-ea85-428d-abca-cafdf2b0cf1d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:30.823371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb451de9-ea85-428d-abca-cafdf2b0cf1d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-fb451de9-ea85-428d-abca-cafdf2b0cf1d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:30.848622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:30.849747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:30.850743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:30.850743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:30.850743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:30.850743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:30.850743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:30.850743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:30.850743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:30.850743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:30.850743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 16:19:30.850743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:30.850743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 389/789] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:19:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:30.858282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb451de9-ea85-428d-abca-cafdf2b0cf1d None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:30.906128 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:30.948897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b7ede-5ea0-4292-9650-442c67701bf8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:30.985965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb451de9-ea85-428d-abca-cafdf2b0cf1d None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:31.017473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02111a-7aba-4ebb-87f9-e6ea37f01e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:31.018133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02111a-7aba-4ebb-87f9-e6ea37f01e2e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:31.018417 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02111a-7aba-4ebb-87f9-e6ea37f01e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:31.018697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02111a-7aba-4ebb-87f9-e6ea37f01e2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:31.021278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0a793-a1fc-45e7-bcb2-db9ddffd01a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:31.021884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0a793-a1fc-45e7-bcb2-db9ddffd01a3 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:31.022169 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0a793-a1fc-45e7-bcb2-db9ddffd01a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:31.022522 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0a793-a1fc-45e7-bcb2-db9ddffd01a3 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:31.023261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0a793-a1fc-45e7-bcb2-db9ddffd01a3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:31.023564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0a793-a1fc-45e7-bcb2-db9ddffd01a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:31.025020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0a793-a1fc-45e7-bcb2-db9ddffd01a3 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:31.032783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02111a-7aba-4ebb-87f9-e6ea37f01e2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-3b02111a-7aba-4ebb-87f9-e6ea37f01e2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:31.040855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:31.042699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:31.043604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 401/790] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:31.044307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b16ab-978c-4715-af74-4d652f8b47d6 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:31.044307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b16ab-978c-4715-af74-4d652f8b47d6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:31.044438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b16ab-978c-4715-af74-4d652f8b47d6 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:31.048887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 402/791] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:31.049016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 390/792] 10.222.0.55 () {56 vars in 1135 bytes} [Fri Aug 19 16:19:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:31.050159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b16ab-978c-4715-af74-4d652f8b47d6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:31.050923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9b92f-cea8-4d27-b323-05e444b13b32 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:31.051358 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9b92f-cea8-4d27-b323-05e444b13b32 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:31.051766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9b92f-cea8-4d27-b323-05e444b13b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:31.071342 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-37a9b92f-cea8-4d27-b323-05e444b13b32 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:31.072609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-37a9b92f-cea8-4d27-b323-05e444b13b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:31.093343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:31.146674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 391/793] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:31.147773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9b92f-cea8-4d27-b323-05e444b13b32 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:31.169114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:31.216326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b16ab-978c-4715-af74-4d652f8b47d6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:31.290478 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bba260-2dfe-4a2a-b26d-b55e25ce6e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:31.291387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bba260-2dfe-4a2a-b26d-b55e25ce6e41 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:31.291832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bba260-2dfe-4a2a-b26d-b55e25ce6e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:31.292260 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bba260-2dfe-4a2a-b26d-b55e25ce6e41 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:31.293102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bba260-2dfe-4a2a-b26d-b55e25ce6e41 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:31.293570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bba260-2dfe-4a2a-b26d-b55e25ce6e41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:31.295084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bba260-2dfe-4a2a-b26d-b55e25ce6e41 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:31.315465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 403/794] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:31.321647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bba260-2dfe-4a2a-b26d-b55e25ce6e41 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:31.338348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9b92f-cea8-4d27-b323-05e444b13b32 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:31.418662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c46ff9-1a59-42fb-9b14-0f72b2c76ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:31.419292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c46ff9-1a59-42fb-9b14-0f72b2c76ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:31.419582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c46ff9-1a59-42fb-9b14-0f72b2c76ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:31.420030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c46ff9-1a59-42fb-9b14-0f72b2c76ba2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:31.431722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c46ff9-1a59-42fb-9b14-0f72b2c76ba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:31.432829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c46ff9-1a59-42fb-9b14-0f72b2c76ba2 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-78c46ff9-1a59-42fb-9b14-0f72b2c76ba2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:31.448102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:31.449060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:31.449928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:31.449928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:31.449928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:31.449928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 392/795] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:31.450554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c46ff9-1a59-42fb-9b14-0f72b2c76ba2 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:31.461887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:31.470030 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bba260-2dfe-4a2a-b26d-b55e25ce6e41 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:31.507774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1a9f6-7972-4b37-918f-e4860635d5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:31.508742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1a9f6-7972-4b37-918f-e4860635d5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:31.509242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1a9f6-7972-4b37-918f-e4860635d5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:31.509735 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1a9f6-7972-4b37-918f-e4860635d5e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:31.522150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1a9f6-7972-4b37-918f-e4860635d5e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:31.524109 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1a9f6-7972-4b37-918f-e4860635d5e0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:31.533866 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-74a1a9f6-7972-4b37-918f-e4860635d5e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:31.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:31.545120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:31.545986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:31.545986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:31.545986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:31.546751 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 404/796] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:31.547512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1a9f6-7972-4b37-918f-e4860635d5e0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:31.589768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c46ff9-1a59-42fb-9b14-0f72b2c76ba2 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:31.628837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f125209-327e-4556-812c-96b18a8c193f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:31.629949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f125209-327e-4556-812c-96b18a8c193f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:31.630288 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f125209-327e-4556-812c-96b18a8c193f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:31.630631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f125209-327e-4556-812c-96b18a8c193f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:31.644855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f125209-327e-4556-812c-96b18a8c193f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-8f125209-327e-4556-812c-96b18a8c193f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:31.651220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:31.652248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:31.653138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:31.653138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:31.653138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:31.653138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:31.653138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:31.653138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:31.653138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:31.653138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:31.653138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:31.653138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:31.653138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:19:31.653138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:31.657012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 393/797] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:19:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:31.658390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f125209-327e-4556-812c-96b18a8c193f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:31.686047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1a9f6-7972-4b37-918f-e4860635d5e0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:31.722430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e3821-469e-4b08-aa9b-308face50417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:31.723832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e3821-469e-4b08-aa9b-308face50417 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:31.724297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e3821-469e-4b08-aa9b-308face50417 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:31.724864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e3821-469e-4b08-aa9b-308face50417 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:31.725673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e3821-469e-4b08-aa9b-308face50417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:31.726056 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e3821-469e-4b08-aa9b-308face50417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:31.727879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e3821-469e-4b08-aa9b-308face50417 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:31.741688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 405/798] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:31.742625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e3821-469e-4b08-aa9b-308face50417 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:31.751485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:31.769873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f125209-327e-4556-812c-96b18a8c193f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:31.794799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:31.810952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f63db-f497-4460-b5ba-846332bef6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:31.811617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f63db-f497-4460-b5ba-846332bef6be None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:31.811891 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f63db-f497-4460-b5ba-846332bef6be None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:31.812221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f63db-f497-4460-b5ba-846332bef6be None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:31.812967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f63db-f497-4460-b5ba-846332bef6be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:31.813263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f63db-f497-4460-b5ba-846332bef6be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:31.814633 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f63db-f497-4460-b5ba-846332bef6be None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:31.829598 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 394/799] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:31.830505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f63db-f497-4460-b5ba-846332bef6be None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:31.857102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e3821-469e-4b08-aa9b-308face50417 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:31.894467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e535-f7aa-499c-b3ea-096687c4e976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:31.895977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e535-f7aa-499c-b3ea-096687c4e976 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:31.896568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e535-f7aa-499c-b3ea-096687c4e976 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:31.897257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e535-f7aa-499c-b3ea-096687c4e976 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:31.898213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279e535-f7aa-499c-b3ea-096687c4e976 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:31.898707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279e535-f7aa-499c-b3ea-096687c4e976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:31.900376 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279e535-f7aa-499c-b3ea-096687c4e976 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:31.914418 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 406/800] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:31.915724 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e535-f7aa-499c-b3ea-096687c4e976 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:31.943393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f63db-f497-4460-b5ba-846332bef6be None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:31.979899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20139a35-114a-49b0-8a60-541a401bde5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:31.980532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139a35-114a-49b0-8a60-541a401bde5e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:31.980858 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139a35-114a-49b0-8a60-541a401bde5e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:31.981136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139a35-114a-49b0-8a60-541a401bde5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:31.988176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20139a35-114a-49b0-8a60-541a401bde5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:31.990794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20139a35-114a-49b0-8a60-541a401bde5e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:31.991292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-20139a35-114a-49b0-8a60-541a401bde5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:32.003271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:32.004300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:32.006030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:32.006030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:32.006030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:32.006030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:32.006030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:32.006030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:32.006030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:32.006030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:32.006449 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 395/801] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:32.012875 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20139a35-114a-49b0-8a60-541a401bde5e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:32.038534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e535-f7aa-499c-b3ea-096687c4e976 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:32.059993 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:32.078537 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dd9b5-07b3-4ec9-becc-12e4a2075490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:32.079513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dd9b5-07b3-4ec9-becc-12e4a2075490 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.080002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dd9b5-07b3-4ec9-becc-12e4a2075490 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.080583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dd9b5-07b3-4ec9-becc-12e4a2075490 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.089396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dd9b5-07b3-4ec9-becc-12e4a2075490 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:32.091261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dd9b5-07b3-4ec9-becc-12e4a2075490 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-b27dd9b5-07b3-4ec9-becc-12e4a2075490 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:32.103845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:32.104839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:32.105765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:32.105765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:32.105765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:32.105765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:32.105765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:32.105765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:32.105765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:32.105765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:32.105765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 16:19:32.105765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:32.108383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 407/802] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Aug 19 16:19:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:32.109625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dd9b5-07b3-4ec9-becc-12e4a2075490 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:32.128029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20139a35-114a-49b0-8a60-541a401bde5e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:32.166104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7d4f2-1177-4f43-b90f-681c76c39b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:32.166744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7d4f2-1177-4f43-b90f-681c76c39b6c None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.167049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7d4f2-1177-4f43-b90f-681c76c39b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.167325 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7d4f2-1177-4f43-b90f-681c76c39b6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.178865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7d4f2-1177-4f43-b90f-681c76c39b6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:32.180309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7d4f2-1177-4f43-b90f-681c76c39b6c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-e9f7d4f2-1177-4f43-b90f-681c76c39b6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:32.196086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:32.197355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:32.198223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:32.198223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:32.198223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:32.198223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 396/803] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:32.198442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7d4f2-1177-4f43-b90f-681c76c39b6c None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:32.229298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dd9b5-07b3-4ec9-becc-12e4a2075490 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:32.257421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:32.265734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a460f-e5f8-4bfa-a3c7-1771eb09a79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:32.266342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a460f-e5f8-4bfa-a3c7-1771eb09a79b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.266615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a460f-e5f8-4bfa-a3c7-1771eb09a79b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.266888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a460f-e5f8-4bfa-a3c7-1771eb09a79b None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.267574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a460f-e5f8-4bfa-a3c7-1771eb09a79b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:32.267874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a460f-e5f8-4bfa-a3c7-1771eb09a79b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:32.269360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a460f-e5f8-4bfa-a3c7-1771eb09a79b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:32.287116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 408/804] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:32.287116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e224dd-e7cf-46a4-a854-06286ac0787f None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.287116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e224dd-e7cf-46a4-a854-06286ac0787f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.287116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e224dd-e7cf-46a4-a854-06286ac0787f None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.288884 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 409/805] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:32.290340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e224dd-e7cf-46a4-a854-06286ac0787f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:32.304862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:32.315168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7d4f2-1177-4f43-b90f-681c76c39b6c None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:32.351797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b9d03-fbf0-4bd9-882c-26fe9ab4244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:32.352682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b9d03-fbf0-4bd9-882c-26fe9ab4244e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.353063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b9d03-fbf0-4bd9-882c-26fe9ab4244e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.353333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b9d03-fbf0-4bd9-882c-26fe9ab4244e None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.354101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b9d03-fbf0-4bd9-882c-26fe9ab4244e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:32.354396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b9d03-fbf0-4bd9-882c-26fe9ab4244e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:32.355976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b9d03-fbf0-4bd9-882c-26fe9ab4244e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:32.378700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 397/806] 10.222.0.55 () {56 vars in 1149 bytes} [Fri Aug 19 16:19:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:32.379265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b9d03-fbf0-4bd9-882c-26fe9ab4244e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:32.407571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e224dd-e7cf-46a4-a854-06286ac0787f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:32.444707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d068bf6-8241-4b0f-886b-2f8a33d78508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:32.445359 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d068bf6-8241-4b0f-886b-2f8a33d78508 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.445632 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d068bf6-8241-4b0f-886b-2f8a33d78508 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.445903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d068bf6-8241-4b0f-886b-2f8a33d78508 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.446539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d068bf6-8241-4b0f-886b-2f8a33d78508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:32.446837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d068bf6-8241-4b0f-886b-2f8a33d78508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:32.448206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d068bf6-8241-4b0f-886b-2f8a33d78508 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:32.459857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 410/807] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:32.461228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4055d2-b5a1-4543-a4f0-24dca28b0658 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.461513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4055d2-b5a1-4543-a4f0-24dca28b0658 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.461786 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4055d2-b5a1-4543-a4f0-24dca28b0658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.480619 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ac4055d2-b5a1-4543-a4f0-24dca28b0658 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:32.481937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ac4055d2-b5a1-4543-a4f0-24dca28b0658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:32.490601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b9d03-fbf0-4bd9-882c-26fe9ab4244e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:32.524652 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fb0d2-3d8c-411d-aa0d-c0769e8f7f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:32.525342 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fb0d2-3d8c-411d-aa0d-c0769e8f7f5c None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.525623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fb0d2-3d8c-411d-aa0d-c0769e8f7f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.525893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fb0d2-3d8c-411d-aa0d-c0769e8f7f5c None admin] PATH_INFO: `/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/c755f9538e984bc3877112896595e158/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.527800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fb0d2-3d8c-411d-aa0d-c0769e8f7f5c None admin] RBAC: Authorizing `identity:create_grant(project_id=d143cf444f354ec0bf2089dae3d94255, user_id=c755f9538e984bc3877112896595e158, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:32.528961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:32.529290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fb0d2-3d8c-411d-aa0d-c0769e8f7f5c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:32.542879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 398/808] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:19:32 2022] PUT /identity/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/c755f9538e984bc3877112896595e158/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:19:32.543672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fb0d2-3d8c-411d-aa0d-c0769e8f7f5c None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:32.557716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:32.566208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 411/809] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:32.567571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4055d2-b5a1-4543-a4f0-24dca28b0658 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:32.685199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fb0d2-3d8c-411d-aa0d-c0769e8f7f5c None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:32.689845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4055d2-b5a1-4543-a4f0-24dca28b0658 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:32.729805 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12063394-b953-48be-96ad-71655a5c9553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:32.730446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12063394-b953-48be-96ad-71655a5c9553 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.730732 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12063394-b953-48be-96ad-71655a5c9553 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.731005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12063394-b953-48be-96ad-71655a5c9553 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.735886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaac949-748c-49a0-a1b9-18a9a75351c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:32.736520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaac949-748c-49a0-a1b9-18a9a75351c8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.738200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaac949-748c-49a0-a1b9-18a9a75351c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.738200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaac949-748c-49a0-a1b9-18a9a75351c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.742914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12063394-b953-48be-96ad-71655a5c9553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:32.744324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12063394-b953-48be-96ad-71655a5c9553 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:32.746278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaac949-748c-49a0-a1b9-18a9a75351c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:32.748962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaac949-748c-49a0-a1b9-18a9a75351c8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-12063394-b953-48be-96ad-71655a5c9553 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:32.762150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:32.763160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:32.764030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:32.764030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:32.764030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-2eaac949-748c-49a0-a1b9-18a9a75351c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:32.764209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:32.765107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:32.766047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:32.766047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:32.766047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:32.766047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:32.766047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:32.766047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:32.766047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:32.766047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:32.766047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:19:32.766047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:32.767702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 399/810] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:32.767846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 412/811] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Aug 19 16:19:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:32.769248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f40c6-d3e1-46fc-b926-6a921a5f2565 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.770970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f40c6-d3e1-46fc-b926-6a921a5f2565 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.770970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f40c6-d3e1-46fc-b926-6a921a5f2565 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.771734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaac949-748c-49a0-a1b9-18a9a75351c8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:32.771989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 400/812] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:32.773761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f40c6-d3e1-46fc-b926-6a921a5f2565 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:32.817871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:32.859439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:32.889428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaac949-748c-49a0-a1b9-18a9a75351c8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:32.902518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f40c6-d3e1-46fc-b926-6a921a5f2565 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:32.927362 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e67a5-4fa4-4725-ab07-35c1f11eb2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:32.927902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e67a5-4fa4-4725-ab07-35c1f11eb2fc None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.928179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e67a5-4fa4-4725-ab07-35c1f11eb2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.928456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e67a5-4fa4-4725-ab07-35c1f11eb2fc None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.929245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e67a5-4fa4-4725-ab07-35c1f11eb2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:32.929657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e67a5-4fa4-4725-ab07-35c1f11eb2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:32.931032 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e67a5-4fa4-4725-ab07-35c1f11eb2fc None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:32.936951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9ba2d-08f0-4845-a1e8-5917c5fa3012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:32.937552 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9ba2d-08f0-4845-a1e8-5917c5fa3012 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.937829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9ba2d-08f0-4845-a1e8-5917c5fa3012 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.938100 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9ba2d-08f0-4845-a1e8-5917c5fa3012 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.938819 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9ba2d-08f0-4845-a1e8-5917c5fa3012 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:32.939118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9ba2d-08f0-4845-a1e8-5917c5fa3012 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:32.940527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9ba2d-08f0-4845-a1e8-5917c5fa3012 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:32.950052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 413/813] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:32.950052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33327b-8973-48a0-997d-dc100756190a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:32.950052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33327b-8973-48a0-997d-dc100756190a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:32.950052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33327b-8973-48a0-997d-dc100756190a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:32.963224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 401/814] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Aug 19 16:19:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:32.964022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9ba2d-08f0-4845-a1e8-5917c5fa3012 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:32.970246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7e33327b-8973-48a0-997d-dc100756190a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:32.971446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7e33327b-8973-48a0-997d-dc100756190a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:33.045981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 414/815] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:33.046932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33327b-8973-48a0-997d-dc100756190a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:33.083520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:33.083709 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9ba2d-08f0-4845-a1e8-5917c5fa3012 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:33.112274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:33.132417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da54200-f88b-41cd-bd4b-507fae8df92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:33.133089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da54200-f88b-41cd-bd4b-507fae8df92b None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:33.133372 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da54200-f88b-41cd-bd4b-507fae8df92b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:33.133758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da54200-f88b-41cd-bd4b-507fae8df92b None admin] PATH_INFO: `/v3/services/network` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:33.134355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da54200-f88b-41cd-bd4b-507fae8df92b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:33.135732 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da54200-f88b-41cd-bd4b-507fae8df92b None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-8da54200-f88b-41cd-bd4b-507fae8df92b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:33.150435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:33.151450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 16:19:33.152255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:33.156856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 402/816] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:32 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:33.157691 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da54200-f88b-41cd-bd4b-507fae8df92b None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:33.170221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33327b-8973-48a0-997d-dc100756190a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:33.209186 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682f0f7-0d61-4e80-8a8d-8f1ba47e3e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:33.210147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682f0f7-0d61-4e80-8a8d-8f1ba47e3e6e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:33.210711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682f0f7-0d61-4e80-8a8d-8f1ba47e3e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:33.211182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682f0f7-0d61-4e80-8a8d-8f1ba47e3e6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:33.218705 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682f0f7-0d61-4e80-8a8d-8f1ba47e3e6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:33.220525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682f0f7-0d61-4e80-8a8d-8f1ba47e3e6e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-3682f0f7-0d61-4e80-8a8d-8f1ba47e3e6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:33.234645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:33.235999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:33.236886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:33.236886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:33.236886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:33.236886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:33.236886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:33.236886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:33.236886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:33.236886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:33.244463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 415/817] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:33.245602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682f0f7-0d61-4e80-8a8d-8f1ba47e3e6e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:33.279081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da54200-f88b-41cd-bd4b-507fae8df92b None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:33.323657 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96b70f-3250-4f4d-9afb-6a0ece91f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:33.324264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96b70f-3250-4f4d-9afb-6a0ece91f487 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:33.324546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96b70f-3250-4f4d-9afb-6a0ece91f487 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:33.324843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96b70f-3250-4f4d-9afb-6a0ece91f487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:33.343402 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:33.356906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682f0f7-0d61-4e80-8a8d-8f1ba47e3e6e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:33.382418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:33.394269 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b47ad6-b093-41ff-928d-7f2b71b2ca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:33.394895 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b47ad6-b093-41ff-928d-7f2b71b2ca76 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:33.395694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b47ad6-b093-41ff-928d-7f2b71b2ca76 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:33.395969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b47ad6-b093-41ff-928d-7f2b71b2ca76 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:33.396680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b47ad6-b093-41ff-928d-7f2b71b2ca76 None admin] RBAC: Adding query filter params (name=network) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:33.397046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b47ad6-b093-41ff-928d-7f2b71b2ca76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:33.398413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b47ad6-b093-41ff-928d-7f2b71b2ca76 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:33.400813 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96b70f-3250-4f4d-9afb-6a0ece91f487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:33.402263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96b70f-3250-4f4d-9afb-6a0ece91f487 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:33.407587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 416/818] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:33.407587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b47ad6-b093-41ff-928d-7f2b71b2ca76 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:33.508640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 403/819] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:19:33.514431 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96b70f-3250-4f4d-9afb-6a0ece91f487 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:33.515974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b47ad6-b093-41ff-928d-7f2b71b2ca76 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:33.554519 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc905da5-3dbf-4849-a612-f128c134a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:33.555133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc905da5-3dbf-4849-a612-f128c134a53f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:33.555414 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc905da5-3dbf-4849-a612-f128c134a53f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:33.555691 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc905da5-3dbf-4849-a612-f128c134a53f None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:33.556546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc905da5-3dbf-4849-a612-f128c134a53f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:33.556874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc905da5-3dbf-4849-a612-f128c134a53f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:33.559112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc905da5-3dbf-4849-a612-f128c134a53f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:33.572721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 417/820] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:33.573571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc905da5-3dbf-4849-a612-f128c134a53f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:33.589106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:33.617718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:33.635885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96b70f-3250-4f4d-9afb-6a0ece91f487 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:33.679742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165f52f-3335-411b-a32b-54479ac4a1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:33.680400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165f52f-3335-411b-a32b-54479ac4a1be None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:33.680678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165f52f-3335-411b-a32b-54479ac4a1be None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:33.680980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165f52f-3335-411b-a32b-54479ac4a1be None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:33.681562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165f52f-3335-411b-a32b-54479ac4a1be None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:33.682922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165f52f-3335-411b-a32b-54479ac4a1be None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:33.686929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc905da5-3dbf-4849-a612-f128c134a53f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-f165f52f-3335-411b-a32b-54479ac4a1be None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:33.692780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:33.693744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 16:19:33.694573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:33.697103 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 404/821] 10.222.0.55 () {56 vars in 1129 bytes} [Fri Aug 19 16:19:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:33.698705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165f52f-3335-411b-a32b-54479ac4a1be None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:33.723818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b6c65-1fcf-4ef8-b2f5-514adebb64a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:33.724456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b6c65-1fcf-4ef8-b2f5-514adebb64a5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:33.724776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b6c65-1fcf-4ef8-b2f5-514adebb64a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:33.725041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b6c65-1fcf-4ef8-b2f5-514adebb64a5 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:33.725665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b6c65-1fcf-4ef8-b2f5-514adebb64a5 None admin] RBAC: Adding query filter params (type=network) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:33.725972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b6c65-1fcf-4ef8-b2f5-514adebb64a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:33.727573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b6c65-1fcf-4ef8-b2f5-514adebb64a5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:33.732428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 418/822] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:33.733195 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b6c65-1fcf-4ef8-b2f5-514adebb64a5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:33.809433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165f52f-3335-411b-a32b-54479ac4a1be None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:33.814318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:33.842375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b6c65-1fcf-4ef8-b2f5-514adebb64a5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:33.844804 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5bc1dc-35d4-4e31-93cc-86c64764bab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:33.845431 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bc1dc-35d4-4e31-93cc-86c64764bab5 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:33.845708 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bc1dc-35d4-4e31-93cc-86c64764bab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:33.845979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bc1dc-35d4-4e31-93cc-86c64764bab5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:33.858193 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5bc1dc-35d4-4e31-93cc-86c64764bab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:33.860010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5bc1dc-35d4-4e31-93cc-86c64764bab5 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:33.868931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-cb5bc1dc-35d4-4e31-93cc-86c64764bab5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:33.875893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:33.876843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:33.877717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:33.877717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:33.877717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:33.884017 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 405/823] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:33.884017 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5bc1dc-35d4-4e31-93cc-86c64764bab5 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:33.886464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c634bd-1a4a-4505-a5db-cb35b53a0419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:33.887893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c634bd-1a4a-4505-a5db-cb35b53a0419 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:33.887893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c634bd-1a4a-4505-a5db-cb35b53a0419 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:33.887893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c634bd-1a4a-4505-a5db-cb35b53a0419 None admin] PATH_INFO: `/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/88694a50b9f04487a521ba6df110d976/roles/3a9f17a4e0074293989007ce1a8b2565` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:33.902959 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c634bd-1a4a-4505-a5db-cb35b53a0419 None admin] RBAC: Authorizing `identity:create_grant(project_id=d143cf444f354ec0bf2089dae3d94255, user_id=88694a50b9f04487a521ba6df110d976, role_id=3a9f17a4e0074293989007ce1a8b2565)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:33.904406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c634bd-1a4a-4505-a5db-cb35b53a0419 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:33.924258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 419/824] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:19:33 2022] PUT /identity/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/88694a50b9f04487a521ba6df110d976/roles/3a9f17a4e0074293989007ce1a8b2565 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:19:33.924258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c634bd-1a4a-4505-a5db-cb35b53a0419 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:34.022972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5bc1dc-35d4-4e31-93cc-86c64764bab5 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:34.054611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c634bd-1a4a-4505-a5db-cb35b53a0419 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:34.074809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26de9c-75a2-4ec9-b442-a504b329525f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:34.075368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26de9c-75a2-4ec9-b442-a504b329525f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:34.075643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26de9c-75a2-4ec9-b442-a504b329525f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:34.075965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26de9c-75a2-4ec9-b442-a504b329525f None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:34.076559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26de9c-75a2-4ec9-b442-a504b329525f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:34.076889 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26de9c-75a2-4ec9-b442-a504b329525f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:34.078268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26de9c-75a2-4ec9-b442-a504b329525f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:34.083310 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 406/825] 10.222.0.55 () {56 vars in 1124 bytes} [Fri Aug 19 16:19:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:34.083949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26de9c-75a2-4ec9-b442-a504b329525f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:34.096343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:34.112995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40731890-a8c8-4373-867a-d2aa40750313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:34.114545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40731890-a8c8-4373-867a-d2aa40750313 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:34.114827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40731890-a8c8-4373-867a-d2aa40750313 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:34.115464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40731890-a8c8-4373-867a-d2aa40750313 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:34.116102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40731890-a8c8-4373-867a-d2aa40750313 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:34.119221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40731890-a8c8-4373-867a-d2aa40750313 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:34.144423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 420/826] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:19:33 2022] POST /identity/v3/endpoints => generated 337 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:34.146077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40731890-a8c8-4373-867a-d2aa40750313 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:34.172957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:34.261670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26de9c-75a2-4ec9-b442-a504b329525f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:34.327725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40731890-a8c8-4373-867a-d2aa40750313 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:34.329837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824222b-d5ec-4330-825d-cb3898a7600b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:34.330843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824222b-d5ec-4330-825d-cb3898a7600b None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:34.331353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824222b-d5ec-4330-825d-cb3898a7600b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:34.331886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824222b-d5ec-4330-825d-cb3898a7600b None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:34.333279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824222b-d5ec-4330-825d-cb3898a7600b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:34.333786 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824222b-d5ec-4330-825d-cb3898a7600b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:34.335368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824222b-d5ec-4330-825d-cb3898a7600b None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:34.350276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 407/827] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:34.357835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824222b-d5ec-4330-825d-cb3898a7600b None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:34.385023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c527b-9951-407c-a436-ea9dc045d49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:34.385693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c527b-9951-407c-a436-ea9dc045d49d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:34.385971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c527b-9951-407c-a436-ea9dc045d49d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:34.386244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c527b-9951-407c-a436-ea9dc045d49d None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:34.386875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c527b-9951-407c-a436-ea9dc045d49d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:34.387174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c527b-9951-407c-a436-ea9dc045d49d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:34.388558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c527b-9951-407c-a436-ea9dc045d49d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:34.394325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 421/828] 10.222.0.55 () {56 vars in 1124 bytes} [Fri Aug 19 16:19:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:34.404988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c527b-9951-407c-a436-ea9dc045d49d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:34.459298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:34.469929 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824222b-d5ec-4330-825d-cb3898a7600b None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:34.497186 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:34.510113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3fc19-e612-4126-ad9b-a0365b053dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:34.510759 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3fc19-e612-4126-ad9b-a0365b053dbc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:34.511041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3fc19-e612-4126-ad9b-a0365b053dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:34.511314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3fc19-e612-4126-ad9b-a0365b053dbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:34.521702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3fc19-e612-4126-ad9b-a0365b053dbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:34.523367 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3fc19-e612-4126-ad9b-a0365b053dbc None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:34.525547 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c527b-9951-407c-a436-ea9dc045d49d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-2cb3fc19-e612-4126-ad9b-a0365b053dbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:34.537249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:34.538178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:34.539051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:34.539051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:34.539051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:34.539051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:34.539051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:34.539051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:34.539051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:34.539051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:34.541759 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 408/829] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:34.546995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3fc19-e612-4126-ad9b-a0365b053dbc None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:34.574999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca87e0-cb5a-4dd9-96f8-d7874c27792a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:34.575628 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca87e0-cb5a-4dd9-96f8-d7874c27792a None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:34.575911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca87e0-cb5a-4dd9-96f8-d7874c27792a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:34.576202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca87e0-cb5a-4dd9-96f8-d7874c27792a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:34.576927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca87e0-cb5a-4dd9-96f8-d7874c27792a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:34.578480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca87e0-cb5a-4dd9-96f8-d7874c27792a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:34.604027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 422/830] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:19:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:34.608821 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed5ff-ec66-4d2a-9fa3-2007324b8221 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:34.608821 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed5ff-ec66-4d2a-9fa3-2007324b8221 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:34.608821 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed5ff-ec66-4d2a-9fa3-2007324b8221 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:34.611780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 423/831] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:34.613315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a4065-46c9-476e-b7c0-80d4269479a7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:34.613647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a4065-46c9-476e-b7c0-80d4269479a7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:34.613932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a4065-46c9-476e-b7c0-80d4269479a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:34.634661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1e7a4065-46c9-476e-b7c0-80d4269479a7 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:34.635953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1e7a4065-46c9-476e-b7c0-80d4269479a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:34.675367 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3fc19-e612-4126-ad9b-a0365b053dbc None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:34.726885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d1d4e-0c87-4cdf-b256-72fe5b330840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:34.727761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d1d4e-0c87-4cdf-b256-72fe5b330840 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:34.728040 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d1d4e-0c87-4cdf-b256-72fe5b330840 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:34.728376 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d1d4e-0c87-4cdf-b256-72fe5b330840 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:34.729121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d1d4e-0c87-4cdf-b256-72fe5b330840 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:34.729424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d1d4e-0c87-4cdf-b256-72fe5b330840 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:34.730899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d1d4e-0c87-4cdf-b256-72fe5b330840 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:34.733098 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 424/832] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:34.749486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 409/833] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:34.750991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a4065-46c9-476e-b7c0-80d4269479a7 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:34.792571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:34.857600 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a4065-46c9-476e-b7c0-80d4269479a7 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:34.892310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad3a22-7899-43d8-9221-25aa160eb3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:34.893042 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad3a22-7899-43d8-9221-25aa160eb3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:34.893329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad3a22-7899-43d8-9221-25aa160eb3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:34.893603 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad3a22-7899-43d8-9221-25aa160eb3a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:34.894329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad3a22-7899-43d8-9221-25aa160eb3a4 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=4739e80619a54b178b9aacec6a42a7f0) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:34.894635 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad3a22-7899-43d8-9221-25aa160eb3a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:34.896624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad3a22-7899-43d8-9221-25aa160eb3a4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:34.908963 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 425/834] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:34 2022] GET /identity/v3/role_assignments?user.id=4739e80619a54b178b9aacec6a42a7f0&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:35.010385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d1d4e-0c87-4cdf-b256-72fe5b330840 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:35.049500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:35.120091 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d1d4e-0c87-4cdf-b256-72fe5b330840 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:35.157891 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a7415-718a-4bde-b979-ffa70b078fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:35.158529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a7415-718a-4bde-b979-ffa70b078fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:35.158806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a7415-718a-4bde-b979-ffa70b078fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:35.159080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a7415-718a-4bde-b979-ffa70b078fc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:35.170063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a7415-718a-4bde-b979-ffa70b078fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-142a7415-718a-4bde-b979-ffa70b078fc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:35.174849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:35.175753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:35.177499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 410/835] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:19:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:35.184605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad3a22-7899-43d8-9221-25aa160eb3a4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:35.285336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:35.294088 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad3a22-7899-43d8-9221-25aa160eb3a4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:35.330762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5527c-7d6d-45eb-8884-78fa70cb27cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:35.331380 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5527c-7d6d-45eb-8884-78fa70cb27cb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:35.331660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5527c-7d6d-45eb-8884-78fa70cb27cb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:35.331952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5527c-7d6d-45eb-8884-78fa70cb27cb None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:35.332625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5527c-7d6d-45eb-8884-78fa70cb27cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:35.332966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5527c-7d6d-45eb-8884-78fa70cb27cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:35.334338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5527c-7d6d-45eb-8884-78fa70cb27cb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:35.343668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 426/836] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:35.349849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a7415-718a-4bde-b979-ffa70b078fc7 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:35.457019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:35.463214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a7415-718a-4bde-b979-ffa70b078fc7 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:35.498984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fba83-c458-4bd8-a1ed-de929348d413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:35.499610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fba83-c458-4bd8-a1ed-de929348d413 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:35.499890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fba83-c458-4bd8-a1ed-de929348d413 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:35.500163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fba83-c458-4bd8-a1ed-de929348d413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:35.511170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fba83-c458-4bd8-a1ed-de929348d413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:35.512670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fba83-c458-4bd8-a1ed-de929348d413 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-7f1fba83-c458-4bd8-a1ed-de929348d413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:35.526999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:35.527986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:35.528914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:35.528914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:35.528914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:35.528914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 411/837] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:35.536037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5527c-7d6d-45eb-8884-78fa70cb27cb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:35.644469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5527c-7d6d-45eb-8884-78fa70cb27cb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:35.680564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7160e-62f8-4b86-8d07-ed0a0b8f42d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:35.681220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7160e-62f8-4b86-8d07-ed0a0b8f42d2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:35.681502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7160e-62f8-4b86-8d07-ed0a0b8f42d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:35.681777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7160e-62f8-4b86-8d07-ed0a0b8f42d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:35.682425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7160e-62f8-4b86-8d07-ed0a0b8f42d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:35.682728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7160e-62f8-4b86-8d07-ed0a0b8f42d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:35.684107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7160e-62f8-4b86-8d07-ed0a0b8f42d2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:35.696422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 427/838] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:35.702002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fba83-c458-4bd8-a1ed-de929348d413 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:35.807426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fba83-c458-4bd8-a1ed-de929348d413 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:35.843223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fc335-bc48-4906-bd6f-372e88f50623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:35.843871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fc335-bc48-4906-bd6f-372e88f50623 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:35.844150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fc335-bc48-4906-bd6f-372e88f50623 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:35.844423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fc335-bc48-4906-bd6f-372e88f50623 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:35.853634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fc335-bc48-4906-bd6f-372e88f50623 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:35.855086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fc335-bc48-4906-bd6f-372e88f50623 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-c95fc335-bc48-4906-bd6f-372e88f50623 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:35.866604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:35.867547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:35.868553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:35.868553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:35.868553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:35.868553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:35.868553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:35.868553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:35.868553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:35.868553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:35.868553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 16:19:35.868553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:35.868553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 412/839] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:19:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:35.875132 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7160e-62f8-4b86-8d07-ed0a0b8f42d2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:35.891557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:35.980446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7160e-62f8-4b86-8d07-ed0a0b8f42d2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:36.003109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fc3c0-e854-4564-b83f-387298b89f9e None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:36.003405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fc3c0-e854-4564-b83f-387298b89f9e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:36.003661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fc3c0-e854-4564-b83f-387298b89f9e None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:36.005481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 413/840] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:36.010196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f3530-462f-4e9c-ae12-91bfd71648ed None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:36.010476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f3530-462f-4e9c-ae12-91bfd71648ed None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:36.011082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f3530-462f-4e9c-ae12-91bfd71648ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:36.014770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772117-ec14-44b6-8c62-01beaa01acb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:36.015385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772117-ec14-44b6-8c62-01beaa01acb0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:36.015665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772117-ec14-44b6-8c62-01beaa01acb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:36.015946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772117-ec14-44b6-8c62-01beaa01acb0 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:36.016675 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f772117-ec14-44b6-8c62-01beaa01acb0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:36.017140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f772117-ec14-44b6-8c62-01beaa01acb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:36.018701 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f772117-ec14-44b6-8c62-01beaa01acb0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:36.029550 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-921f3530-462f-4e9c-ae12-91bfd71648ed None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:36.030745 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-921f3530-462f-4e9c-ae12-91bfd71648ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:36.032780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:36.039709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 428/841] 10.222.0.55 () {56 vars in 1135 bytes} [Fri Aug 19 16:19:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:36.045142 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772117-ec14-44b6-8c62-01beaa01acb0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:36.119526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 414/842] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:36.130936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:36.162459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772117-ec14-44b6-8c62-01beaa01acb0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:36.200895 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ba4f4-11c5-4d90-9321-605876059c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:36.201662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ba4f4-11c5-4d90-9321-605876059c0e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:36.201824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ba4f4-11c5-4d90-9321-605876059c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:36.202035 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ba4f4-11c5-4d90-9321-605876059c0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:36.272830 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ba4f4-11c5-4d90-9321-605876059c0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:36.274296 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ba4f4-11c5-4d90-9321-605876059c0e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:36.343744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:36.375832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 429/843] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 16:19:36.381736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f3530-462f-4e9c-ae12-91bfd71648ed None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:36.386806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ba4f4-11c5-4d90-9321-605876059c0e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:36.494421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f3530-462f-4e9c-ae12-91bfd71648ed None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:36.500176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ba4f4-11c5-4d90-9321-605876059c0e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:36.531346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a418bad-b9ff-4f3f-a4fd-4d3caf841d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:36.532080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a418bad-b9ff-4f3f-a4fd-4d3caf841d0a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:36.532473 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a418bad-b9ff-4f3f-a4fd-4d3caf841d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:36.532774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a418bad-b9ff-4f3f-a4fd-4d3caf841d0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:36.540400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:36.544087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d185ad-af8f-47e5-b438-086bee550cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:36.544353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a418bad-b9ff-4f3f-a4fd-4d3caf841d0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:36.545192 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d185ad-af8f-47e5-b438-086bee550cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:36.545654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d185ad-af8f-47e5-b438-086bee550cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:36.545829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a418bad-b9ff-4f3f-a4fd-4d3caf841d0a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:36.546375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d185ad-af8f-47e5-b438-086bee550cd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:36.560641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d185ad-af8f-47e5-b438-086bee550cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-3a418bad-b9ff-4f3f-a4fd-4d3caf841d0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:36.562934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:36.563883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:36.564773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:36.564773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:36.564773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:36.569182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 415/844] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c7d185ad-af8f-47e5-b438-086bee550cd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:36.569424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:36.570337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:36.571174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:36.571174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:36.571174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:36.571174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:36.571174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:36.571174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:36.571174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:36.571174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:36.571174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:36.571174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:36.571174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:19:36.571174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:36.571929 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a418bad-b9ff-4f3f-a4fd-4d3caf841d0a None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:36.573631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 430/845] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:19:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:36.579033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d185ad-af8f-47e5-b438-086bee550cd1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:36.618493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:36.704605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a418bad-b9ff-4f3f-a4fd-4d3caf841d0a None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:36.745193 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee97605-2e80-4255-b69d-dce057567581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:36.746250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee97605-2e80-4255-b69d-dce057567581 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:36.746740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee97605-2e80-4255-b69d-dce057567581 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:36.747219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee97605-2e80-4255-b69d-dce057567581 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:36.748152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee97605-2e80-4255-b69d-dce057567581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:36.748663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee97605-2e80-4255-b69d-dce057567581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:36.750309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee97605-2e80-4255-b69d-dce057567581 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:36.765383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 416/846] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:36.767000 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b387e-421a-49bb-a640-19f880eb4f89 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:36.767491 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b387e-421a-49bb-a640-19f880eb4f89 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:36.767970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b387e-421a-49bb-a640-19f880eb4f89 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:36.769757 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 417/847] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:36.774302 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b387e-421a-49bb-a640-19f880eb4f89 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:36.814040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d185ad-af8f-47e5-b438-086bee550cd1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:36.834546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:36.857588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e70c98-76b2-4677-995b-a1ba1f855fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:36.858528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e70c98-76b2-4677-995b-a1ba1f855fef None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:36.859040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e70c98-76b2-4677-995b-a1ba1f855fef None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:36.859536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e70c98-76b2-4677-995b-a1ba1f855fef None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:36.860401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e70c98-76b2-4677-995b-a1ba1f855fef None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:36.860951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e70c98-76b2-4677-995b-a1ba1f855fef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:36.863090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e70c98-76b2-4677-995b-a1ba1f855fef None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:36.877472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 431/848] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:19:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:36.877472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d5533-bcb9-4b8a-9086-c6f5de168a30 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:36.877472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d5533-bcb9-4b8a-9086-c6f5de168a30 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:36.877472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d5533-bcb9-4b8a-9086-c6f5de168a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:36.892501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b387e-421a-49bb-a640-19f880eb4f89 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:36.893049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bb0d5533-bcb9-4b8a-9086-c6f5de168a30 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:36.894259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bb0d5533-bcb9-4b8a-9086-c6f5de168a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:36.930169 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcd394-4d2e-469f-9832-c4063610b10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:36.930905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd394-4d2e-469f-9832-c4063610b10f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:36.931195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd394-4d2e-469f-9832-c4063610b10f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:36.931474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd394-4d2e-469f-9832-c4063610b10f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:36.938407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcd394-4d2e-469f-9832-c4063610b10f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:36.939845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcd394-4d2e-469f-9832-c4063610b10f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-61fcd394-4d2e-469f-9832-c4063610b10f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:36.951391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:36.952349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:36.953260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:36.953260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:36.953260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:36.953260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:36.953260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:36.953260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:36.953260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:36.953260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:36.953260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 418/849] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:36.953260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcd394-4d2e-469f-9832-c4063610b10f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:36.975164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 432/850] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:36.976439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d5533-bcb9-4b8a-9086-c6f5de168a30 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:37.008458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:37.072459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcd394-4d2e-469f-9832-c4063610b10f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:37.084165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:37.084309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d5533-bcb9-4b8a-9086-c6f5de168a30 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:37.115498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c1c7e-5ec3-4033-8e93-d7a3ed1ffa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:37.116407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c1c7e-5ec3-4033-8e93-d7a3ed1ffa3e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:37.116693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c1c7e-5ec3-4033-8e93-d7a3ed1ffa3e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:37.117033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c1c7e-5ec3-4033-8e93-d7a3ed1ffa3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:37.120292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17de9f-a701-4221-b892-47dbc7980611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:37.120924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17de9f-a701-4221-b892-47dbc7980611 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:37.121204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17de9f-a701-4221-b892-47dbc7980611 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:37.121475 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17de9f-a701-4221-b892-47dbc7980611 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:37.122158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17de9f-a701-4221-b892-47dbc7980611 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:37.122459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17de9f-a701-4221-b892-47dbc7980611 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:37.124264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17de9f-a701-4221-b892-47dbc7980611 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:37.128098 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c1c7e-5ec3-4033-8e93-d7a3ed1ffa3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:37.129750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c1c7e-5ec3-4033-8e93-d7a3ed1ffa3e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:37.141086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 433/851] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:37.141086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38d7ab-5c99-419a-8194-e8b2b076bda2 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:37.141086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38d7ab-5c99-419a-8194-e8b2b076bda2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:37.141086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38d7ab-5c99-419a-8194-e8b2b076bda2 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:37.143593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 434/852] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:37.144357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38d7ab-5c99-419a-8194-e8b2b076bda2 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-d42c1c7e-5ec3-4033-8e93-d7a3ed1ffa3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:37.151055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:37.152145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:37.153211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:37.153211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:37.153211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:37.158353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 419/853] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:37.158353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b8326-a818-462c-8cab-991db6b5c47d None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:37.158353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b8326-a818-462c-8cab-991db6b5c47d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:37.158353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b8326-a818-462c-8cab-991db6b5c47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:37.180060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-295b8326-a818-462c-8cab-991db6b5c47d None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:37.182167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-295b8326-a818-462c-8cab-991db6b5c47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:37.233690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:37.251088 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38d7ab-5c99-419a-8194-e8b2b076bda2 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:37.259068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 420/854] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:37.259902 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b8326-a818-462c-8cab-991db6b5c47d None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:37.296046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb8b0d-c1b5-48bb-afbc-1a7fbc8a6f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:37.296702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb8b0d-c1b5-48bb-afbc-1a7fbc8a6f66 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:37.297006 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb8b0d-c1b5-48bb-afbc-1a7fbc8a6f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:37.297278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb8b0d-c1b5-48bb-afbc-1a7fbc8a6f66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:37.298035 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb8b0d-c1b5-48bb-afbc-1a7fbc8a6f66 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=c755f9538e984bc3877112896595e158) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:37.298461 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb8b0d-c1b5-48bb-afbc-1a7fbc8a6f66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:37.299926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb8b0d-c1b5-48bb-afbc-1a7fbc8a6f66 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:37.316790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 435/855] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:37 2022] GET /identity/v3/role_assignments?user.id=c755f9538e984bc3877112896595e158&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:37.316790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb8b0d-c1b5-48bb-afbc-1a7fbc8a6f66 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:37.383394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:37.394074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b8326-a818-462c-8cab-991db6b5c47d None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:37.436104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e9b1-d00b-4922-afc9-942cfaa5cddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:37.436746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e9b1-d00b-4922-afc9-942cfaa5cddb None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:37.437343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e9b1-d00b-4922-afc9-942cfaa5cddb None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:37.437529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e9b1-d00b-4922-afc9-942cfaa5cddb None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:37.438257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4e9b1-d00b-4922-afc9-942cfaa5cddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:37.438624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4e9b1-d00b-4922-afc9-942cfaa5cddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:37.440230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4e9b1-d00b-4922-afc9-942cfaa5cddb None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:37.454081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 421/856] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:37.455221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb8b0d-c1b5-48bb-afbc-1a7fbc8a6f66 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:37.459862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e9b1-d00b-4922-afc9-942cfaa5cddb None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:37.495672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddccc6-71ec-48dd-81e4-a327f3d42083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:37.496628 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddccc6-71ec-48dd-81e4-a327f3d42083 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:37.496963 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddccc6-71ec-48dd-81e4-a327f3d42083 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:37.497242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddccc6-71ec-48dd-81e4-a327f3d42083 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:37.498104 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddccc6-71ec-48dd-81e4-a327f3d42083 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:37.499498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddccc6-71ec-48dd-81e4-a327f3d42083 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-88ddccc6-71ec-48dd-81e4-a327f3d42083 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:37.508812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:37.509800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:37.510638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 436/857] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:19:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:37.515641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddccc6-71ec-48dd-81e4-a327f3d42083 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:37.534171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:37.586251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e9b1-d00b-4922-afc9-942cfaa5cddb None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:37.623060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f5d96-b3eb-4687-b3ad-ac165f3a805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:37.623695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f5d96-b3eb-4687-b3ad-ac165f3a805c None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:37.623979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f5d96-b3eb-4687-b3ad-ac165f3a805c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:37.624251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f5d96-b3eb-4687-b3ad-ac165f3a805c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:37.632374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddccc6-71ec-48dd-81e4-a327f3d42083 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:37.633513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f5d96-b3eb-4687-b3ad-ac165f3a805c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:37.634945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f5d96-b3eb-4687-b3ad-ac165f3a805c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-6c0f5d96-b3eb-4687-b3ad-ac165f3a805c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:37.646925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:37.648113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:37.649263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:37.649263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:37.649263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:37.649263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:37.649263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:37.649263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:37.649263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:37.649263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:37.649263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 16:19:37.649263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:37.649263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 422/858] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Aug 19 16:19:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:37.649855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f5d96-b3eb-4687-b3ad-ac165f3a805c None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:37.669414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:37.679159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df13808-a5b3-4ddd-a9fc-6eacaf5cb165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:37.679770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df13808-a5b3-4ddd-a9fc-6eacaf5cb165 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:37.680045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df13808-a5b3-4ddd-a9fc-6eacaf5cb165 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:37.680313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df13808-a5b3-4ddd-a9fc-6eacaf5cb165 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:37.680966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df13808-a5b3-4ddd-a9fc-6eacaf5cb165 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:37.681286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df13808-a5b3-4ddd-a9fc-6eacaf5cb165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:37.682619 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df13808-a5b3-4ddd-a9fc-6eacaf5cb165 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:37.687184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 437/859] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:19:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:37.688262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df13808-a5b3-4ddd-a9fc-6eacaf5cb165 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:37.768955 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:37.772631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f5d96-b3eb-4687-b3ad-ac165f3a805c None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:37.793364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df13808-a5b3-4ddd-a9fc-6eacaf5cb165 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:37.808016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b58fc-b3a7-4b24-99f9-2b437f649608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:37.808648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b58fc-b3a7-4b24-99f9-2b437f649608 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:37.808957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b58fc-b3a7-4b24-99f9-2b437f649608 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:37.809231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b58fc-b3a7-4b24-99f9-2b437f649608 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:37.809802 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b58fc-b3a7-4b24-99f9-2b437f649608 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:37.811158 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b58fc-b3a7-4b24-99f9-2b437f649608 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-7b5b58fc-b3a7-4b24-99f9-2b437f649608 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:37.820476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:37.821546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:37.822930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 423/860] 10.222.0.55 () {56 vars in 1132 bytes} [Fri Aug 19 16:19:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:37.823729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b58fc-b3a7-4b24-99f9-2b437f649608 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:37.837364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164a6f3-d32b-4646-9828-713241255e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:37.837982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164a6f3-d32b-4646-9828-713241255e29 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:37.838260 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164a6f3-d32b-4646-9828-713241255e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:37.838529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164a6f3-d32b-4646-9828-713241255e29 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:37.839252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164a6f3-d32b-4646-9828-713241255e29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:37.839550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164a6f3-d32b-4646-9828-713241255e29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:37.840988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164a6f3-d32b-4646-9828-713241255e29 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:37.863961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 438/861] 10.222.0.55 () {56 vars in 1149 bytes} [Fri Aug 19 16:19:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:37.863961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164a6f3-d32b-4646-9828-713241255e29 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:37.905079 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:37.932212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b58fc-b3a7-4b24-99f9-2b437f649608 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:37.967707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ee2b6-ed90-40db-940e-7f9215ed1f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:37.968233 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ee2b6-ed90-40db-940e-7f9215ed1f08 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:37.968511 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ee2b6-ed90-40db-940e-7f9215ed1f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:37.968806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ee2b6-ed90-40db-940e-7f9215ed1f08 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:37.969425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ee2b6-ed90-40db-940e-7f9215ed1f08 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:37.969725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ee2b6-ed90-40db-940e-7f9215ed1f08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:37.971108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ee2b6-ed90-40db-940e-7f9215ed1f08 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:37.975617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 424/862] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:19:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:37.976475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ee2b6-ed90-40db-940e-7f9215ed1f08 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:37.977198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164a6f3-d32b-4646-9828-713241255e29 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:38.033082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e24e-e3ff-4e9c-877b-367ce8a80a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:38.034326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e24e-e3ff-4e9c-877b-367ce8a80a33 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:38.035323 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e24e-e3ff-4e9c-877b-367ce8a80a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:38.035978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e24e-e3ff-4e9c-877b-367ce8a80a33 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:38.037151 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69e24e-e3ff-4e9c-877b-367ce8a80a33 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:38.037857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69e24e-e3ff-4e9c-877b-367ce8a80a33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:38.039890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69e24e-e3ff-4e9c-877b-367ce8a80a33 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:38.041386 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:38.048266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 439/863] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Aug 19 16:19:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:38.053562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e24e-e3ff-4e9c-877b-367ce8a80a33 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:38.099594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ee2b6-ed90-40db-940e-7f9215ed1f08 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:38.144099 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01edec9-1b82-4e47-9ce6-3ed88375ba0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:38.145074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01edec9-1b82-4e47-9ce6-3ed88375ba0d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:38.145500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01edec9-1b82-4e47-9ce6-3ed88375ba0d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:38.145903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01edec9-1b82-4e47-9ce6-3ed88375ba0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:38.157906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:38.194479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e24e-e3ff-4e9c-877b-367ce8a80a33 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:38.227625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01edec9-1b82-4e47-9ce6-3ed88375ba0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:38.229546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01edec9-1b82-4e47-9ce6-3ed88375ba0d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:38.235795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da58a97-6b5e-481e-8539-82c19b3932f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:38.236528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da58a97-6b5e-481e-8539-82c19b3932f8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:38.236958 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da58a97-6b5e-481e-8539-82c19b3932f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:38.237505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da58a97-6b5e-481e-8539-82c19b3932f8 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:38.238256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da58a97-6b5e-481e-8539-82c19b3932f8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:38.239169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da58a97-6b5e-481e-8539-82c19b3932f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:38.240823 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da58a97-6b5e-481e-8539-82c19b3932f8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:38.246867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 440/864] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Aug 19 16:19:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:38.348832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 425/865] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:37 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:19:38.354991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da58a97-6b5e-481e-8539-82c19b3932f8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:38.424746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:38.471467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da58a97-6b5e-481e-8539-82c19b3932f8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:38.511025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f2bd6-897b-41f0-bb1a-9756d2e69ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:38.511668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f2bd6-897b-41f0-bb1a-9756d2e69ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:38.511948 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f2bd6-897b-41f0-bb1a-9756d2e69ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:38.512221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f2bd6-897b-41f0-bb1a-9756d2e69ec8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:38.522645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f2bd6-897b-41f0-bb1a-9756d2e69ec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:38.524092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f2bd6-897b-41f0-bb1a-9756d2e69ec8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-759f2bd6-897b-41f0-bb1a-9756d2e69ec8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:38.542910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:38.543865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:38.544768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:38.544768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:38.544768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:38.544768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 441/866] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:38.549686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01edec9-1b82-4e47-9ce6-3ed88375ba0d None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:38.592795 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:38.655071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01edec9-1b82-4e47-9ce6-3ed88375ba0d None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:38.690955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbe1c4-6050-43e7-b778-5f92c9027356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:38.691613 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbe1c4-6050-43e7-b778-5f92c9027356 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:38.691934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbe1c4-6050-43e7-b778-5f92c9027356 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:38.692209 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbe1c4-6050-43e7-b778-5f92c9027356 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:38.692876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbe1c4-6050-43e7-b778-5f92c9027356 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:38.693188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbe1c4-6050-43e7-b778-5f92c9027356 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:38.694542 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbe1c4-6050-43e7-b778-5f92c9027356 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:38.710833 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 426/867] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:38.716222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f2bd6-897b-41f0-bb1a-9756d2e69ec8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:38.820187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f2bd6-897b-41f0-bb1a-9756d2e69ec8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:38.834588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:38.860065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611570fc-9915-4844-be28-9bf5e2c7ca84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:38.861019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611570fc-9915-4844-be28-9bf5e2c7ca84 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:38.861450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611570fc-9915-4844-be28-9bf5e2c7ca84 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:38.861868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611570fc-9915-4844-be28-9bf5e2c7ca84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:38.869174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611570fc-9915-4844-be28-9bf5e2c7ca84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:38.870801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611570fc-9915-4844-be28-9bf5e2c7ca84 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-611570fc-9915-4844-be28-9bf5e2c7ca84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:38.882471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:38.883592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:38.884677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:38.884677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:38.884677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:38.884677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:38.884677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:38.884677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:38.884677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:38.884677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:38.886712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 442/868] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:38.896778 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbe1c4-6050-43e7-b778-5f92c9027356 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:38.998521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:39.002035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbe1c4-6050-43e7-b778-5f92c9027356 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:39.038225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950e68b-843b-4b84-a745-bf7434d187c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:39.038886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950e68b-843b-4b84-a745-bf7434d187c3 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:39.039166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950e68b-843b-4b84-a745-bf7434d187c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:39.039436 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950e68b-843b-4b84-a745-bf7434d187c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:39.040120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950e68b-843b-4b84-a745-bf7434d187c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:39.040417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950e68b-843b-4b84-a745-bf7434d187c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:39.041884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950e68b-843b-4b84-a745-bf7434d187c3 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:39.054924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 427/869] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:39.060472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611570fc-9915-4844-be28-9bf5e2c7ca84 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:39.168969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611570fc-9915-4844-be28-9bf5e2c7ca84 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:39.204302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3a74c-93df-4288-8eb3-63b9ab72140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:39.204933 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3a74c-93df-4288-8eb3-63b9ab72140f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:39.205213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3a74c-93df-4288-8eb3-63b9ab72140f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:39.205484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3a74c-93df-4288-8eb3-63b9ab72140f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:39.206264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3a74c-93df-4288-8eb3-63b9ab72140f None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=88694a50b9f04487a521ba6df110d976) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:39.206575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3a74c-93df-4288-8eb3-63b9ab72140f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:39.208626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3a74c-93df-4288-8eb3-63b9ab72140f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:39.215028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 443/870] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:39 2022] GET /identity/v3/role_assignments?user.id=88694a50b9f04487a521ba6df110d976&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:39.372361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263a934-557c-4f88-8233-6fc80974261a None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:39.372986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263a934-557c-4f88-8233-6fc80974261a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:39.372986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263a934-557c-4f88-8233-6fc80974261a None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:39.374575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 428/871] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:39.380646 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ad1b9-8f5a-4eca-a0b2-2944646fda8f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:39.381075 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ad1b9-8f5a-4eca-a0b2-2944646fda8f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:39.381259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ad1b9-8f5a-4eca-a0b2-2944646fda8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:39.401940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-133ad1b9-8f5a-4eca-a0b2-2944646fda8f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:39.403275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-133ad1b9-8f5a-4eca-a0b2-2944646fda8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:39.437450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:39.481128 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 444/872] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:39.746834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263a934-557c-4f88-8233-6fc80974261a None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:39.853133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263a934-557c-4f88-8233-6fc80974261a None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:39.889287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c68f1-0418-4eeb-a897-96fc7f12d139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:39.890257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c68f1-0418-4eeb-a897-96fc7f12d139 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:39.890731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c68f1-0418-4eeb-a897-96fc7f12d139 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:39.891223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c68f1-0418-4eeb-a897-96fc7f12d139 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:39.892008 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c68f1-0418-4eeb-a897-96fc7f12d139 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:39.893583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c68f1-0418-4eeb-a897-96fc7f12d139 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-e50c68f1-0418-4eeb-a897-96fc7f12d139 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:39.903254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:39.904224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:39.905070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:39.907782 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 429/873] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Aug 19 16:19:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:39.914000 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ad1b9-8f5a-4eca-a0b2-2944646fda8f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:40.016313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ad1b9-8f5a-4eca-a0b2-2944646fda8f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:40.052246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426bf17-8af8-4b23-adef-f3dd7bd20798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:40.053310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426bf17-8af8-4b23-adef-f3dd7bd20798 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:40.053788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426bf17-8af8-4b23-adef-f3dd7bd20798 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:40.054296 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426bf17-8af8-4b23-adef-f3dd7bd20798 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:40.054518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd6142-99d8-418c-8e82-a1e2f15dadef None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:40.054815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd6142-99d8-418c-8e82-a1e2f15dadef None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:40.055096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd6142-99d8-418c-8e82-a1e2f15dadef None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:40.055803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426bf17-8af8-4b23-adef-f3dd7bd20798 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:40.056297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426bf17-8af8-4b23-adef-f3dd7bd20798 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:40.056688 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 430/874] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:40.058163 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426bf17-8af8-4b23-adef-f3dd7bd20798 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:40.061566 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16eab1-9545-43cd-94fc-daf305e16cbe None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:40.061865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16eab1-9545-43cd-94fc-daf305e16cbe None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:40.062133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16eab1-9545-43cd-94fc-daf305e16cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:40.064103 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 445/875] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Aug 19 16:19:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:40.070712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426bf17-8af8-4b23-adef-f3dd7bd20798 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:40.081382 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-0e16eab1-9545-43cd-94fc-daf305e16cbe None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:40.083314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0e16eab1-9545-43cd-94fc-daf305e16cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:40.102453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:40.105129 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:40.165063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 431/876] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:40.177976 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426bf17-8af8-4b23-adef-f3dd7bd20798 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:40.212588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd495650-9159-4117-a698-582b1f629db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:40.213238 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd495650-9159-4117-a698-582b1f629db8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:40.213518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd495650-9159-4117-a698-582b1f629db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:40.213793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd495650-9159-4117-a698-582b1f629db8 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:40.214422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd495650-9159-4117-a698-582b1f629db8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:40.214725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd495650-9159-4117-a698-582b1f629db8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:40.216092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd495650-9159-4117-a698-582b1f629db8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:40.220832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 446/877] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Aug 19 16:19:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:40.244871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864774a5-06e1-4f3d-8a90-979af03cfaa8 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:40.245157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864774a5-06e1-4f3d-8a90-979af03cfaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:40.245433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864774a5-06e1-4f3d-8a90-979af03cfaa8 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:40.248272 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 432/878] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:40.252896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80a479-6190-41f6-ae7e-3320a846deca None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:40.253184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80a479-6190-41f6-ae7e-3320a846deca None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:40.253458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80a479-6190-41f6-ae7e-3320a846deca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:40.273515 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ad80a479-6190-41f6-ae7e-3320a846deca None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:40.274680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ad80a479-6190-41f6-ae7e-3320a846deca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:40.351101 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 447/879] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:40.433490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864774a5-06e1-4f3d-8a90-979af03cfaa8 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:40.539619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864774a5-06e1-4f3d-8a90-979af03cfaa8 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:40.575801 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8cf04-60c2-4a3b-99b2-5011b3d713b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:40.576469 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8cf04-60c2-4a3b-99b2-5011b3d713b1 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:40.576770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8cf04-60c2-4a3b-99b2-5011b3d713b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:40.577051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8cf04-60c2-4a3b-99b2-5011b3d713b1 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:40.577708 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8cf04-60c2-4a3b-99b2-5011b3d713b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:40.579122 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8cf04-60c2-4a3b-99b2-5011b3d713b1 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:40.596893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 433/880] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:19:40 2022] POST /identity/v3/services => generated 251 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:40.624885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80a479-6190-41f6-ae7e-3320a846deca None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:40.665600 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:40.750984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80a479-6190-41f6-ae7e-3320a846deca None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:40.798614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914b158-2b83-44ba-b416-a70eaba86be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:40.799151 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914b158-2b83-44ba-b416-a70eaba86be8 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:40.799427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914b158-2b83-44ba-b416-a70eaba86be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:40.799762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914b158-2b83-44ba-b416-a70eaba86be8 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:40.800402 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914b158-2b83-44ba-b416-a70eaba86be8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:40.802633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914b158-2b83-44ba-b416-a70eaba86be8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:40.822822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 448/881] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:19:40 2022] POST /identity/v3/services => generated 269 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:41.295084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638111ff-9960-473d-a98c-31e0ebf37954 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:41.295815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638111ff-9960-473d-a98c-31e0ebf37954 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:41.296303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638111ff-9960-473d-a98c-31e0ebf37954 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:41.298338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 434/882] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:41.304709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5c8f2-704f-45c1-b1d4-7f550db3e742 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:41.305289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5c8f2-704f-45c1-b1d4-7f550db3e742 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:41.305765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5c8f2-704f-45c1-b1d4-7f550db3e742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:41.325029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-05d5c8f2-704f-45c1-b1d4-7f550db3e742 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:41.326565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-05d5c8f2-704f-45c1-b1d4-7f550db3e742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:41.404524 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 449/883] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:41.674856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638111ff-9960-473d-a98c-31e0ebf37954 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:41.693203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:41.787609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638111ff-9960-473d-a98c-31e0ebf37954 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:41.826478 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4774c-d09b-41c4-9881-76113f7b17dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:41.827117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4774c-d09b-41c4-9881-76113f7b17dd None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:41.827465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4774c-d09b-41c4-9881-76113f7b17dd None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:41.827743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4774c-d09b-41c4-9881-76113f7b17dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:41.838986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4774c-d09b-41c4-9881-76113f7b17dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-ace4774c-d09b-41c4-9881-76113f7b17dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:41.844221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:41.845178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:41.845997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 435/884] 10.222.0.55 () {56 vars in 1120 bytes} [Fri Aug 19 16:19:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:41.854151 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5c8f2-704f-45c1-b1d4-7f550db3e742 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:41.868235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:41.962742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5c8f2-704f-45c1-b1d4-7f550db3e742 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:41.999937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f41d6-dc4b-4d83-a02e-e676e4ee1a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:42.000546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f41d6-dc4b-4d83-a02e-e676e4ee1a84 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.000863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f41d6-dc4b-4d83-a02e-e676e4ee1a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.001137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f41d6-dc4b-4d83-a02e-e676e4ee1a84 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.001800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f41d6-dc4b-4d83-a02e-e676e4ee1a84 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:42.002103 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f41d6-dc4b-4d83-a02e-e676e4ee1a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:42.003467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f41d6-dc4b-4d83-a02e-e676e4ee1a84 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:42.012515 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 450/885] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Aug 19 16:19:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:42.018046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4774c-d09b-41c4-9881-76113f7b17dd None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:42.104786 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:42.133052 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4774c-d09b-41c4-9881-76113f7b17dd None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:42.169465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482142f7-ce4a-4a60-bc49-ad4611fcc77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:42.170101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482142f7-ce4a-4a60-bc49-ad4611fcc77a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.170381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482142f7-ce4a-4a60-bc49-ad4611fcc77a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.170652 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482142f7-ce4a-4a60-bc49-ad4611fcc77a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.182538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482142f7-ce4a-4a60-bc49-ad4611fcc77a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:42.183990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482142f7-ce4a-4a60-bc49-ad4611fcc77a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-482142f7-ce4a-4a60-bc49-ad4611fcc77a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:42.198826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:42.201010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:42.203039 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:42.203039 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:42.203039 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:42.203039 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 436/886] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:42.205960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f41d6-dc4b-4d83-a02e-e676e4ee1a84 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:42.282925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:42.303685 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaedba-86b0-4668-a18f-d426822bf98a None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.303999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaedba-86b0-4668-a18f-d426822bf98a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.304232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaedba-86b0-4668-a18f-d426822bf98a None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.305866 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 437/887] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:42.310881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf1fa4-069b-4bf9-80f6-a4138edc6d71 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.311169 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf1fa4-069b-4bf9-80f6-a4138edc6d71 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.311451 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf1fa4-069b-4bf9-80f6-a4138edc6d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.317020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f41d6-dc4b-4d83-a02e-e676e4ee1a84 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:42.330263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-c6bf1fa4-069b-4bf9-80f6-a4138edc6d71 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:42.331470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c6bf1fa4-069b-4bf9-80f6-a4138edc6d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:42.352385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400f342-d85a-4328-a654-eeb1d5c2aa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:42.353048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400f342-d85a-4328-a654-eeb1d5c2aa9d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.353326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400f342-d85a-4328-a654-eeb1d5c2aa9d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.353601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400f342-d85a-4328-a654-eeb1d5c2aa9d None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.354244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400f342-d85a-4328-a654-eeb1d5c2aa9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:42.354548 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400f342-d85a-4328-a654-eeb1d5c2aa9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:42.355920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400f342-d85a-4328-a654-eeb1d5c2aa9d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:42.369098 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 451/888] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:42.374080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400f342-d85a-4328-a654-eeb1d5c2aa9d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:42.416943 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 438/889] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:42.478790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400f342-d85a-4328-a654-eeb1d5c2aa9d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:42.508811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:42.514548 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1e32e-432c-4727-874d-5b598ed1c2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:42.515190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1e32e-432c-4727-874d-5b598ed1c2cc None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.515471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1e32e-432c-4727-874d-5b598ed1c2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.515748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1e32e-432c-4727-874d-5b598ed1c2cc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.524136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1e32e-432c-4727-874d-5b598ed1c2cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:42.525632 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1e32e-432c-4727-874d-5b598ed1c2cc None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-69c1e32e-432c-4727-874d-5b598ed1c2cc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:42.538394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:42.540498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:42.542147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:42.542147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:42.542147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:42.542147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:42.542147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:42.542147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:42.542147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:42.542147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:42.542147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 16:19:42.542147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:42.542147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 452/890] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Aug 19 16:19:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:42.545408 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf1fa4-069b-4bf9-80f6-a4138edc6d71 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:42.607457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:42.632222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec85d4-29d5-4787-91b3-921c2808675c None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.632666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec85d4-29d5-4787-91b3-921c2808675c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.633084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec85d4-29d5-4787-91b3-921c2808675c None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.634651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 453/891] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:19:42.639332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef63cc-fed1-41c9-9666-236fe6683273 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.639682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef63cc-fed1-41c9-9666-236fe6683273 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.640050 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef63cc-fed1-41c9-9666-236fe6683273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.668374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4fef63cc-fed1-41c9-9666-236fe6683273 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:42.669133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf1fa4-069b-4bf9-80f6-a4138edc6d71 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:42.670154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4fef63cc-fed1-41c9-9666-236fe6683273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:42.711704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fbd1c-99da-4777-bdbf-9f6ef9c341d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:42.712368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fbd1c-99da-4777-bdbf-9f6ef9c341d7 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.712840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fbd1c-99da-4777-bdbf-9f6ef9c341d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.713260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fbd1c-99da-4777-bdbf-9f6ef9c341d7 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.714172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fbd1c-99da-4777-bdbf-9f6ef9c341d7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:42.714665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fbd1c-99da-4777-bdbf-9f6ef9c341d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:42.716283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fbd1c-99da-4777-bdbf-9f6ef9c341d7 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:42.735595 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 439/892] 10.222.0.55 () {56 vars in 1149 bytes} [Fri Aug 19 16:19:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:42.736657 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fbd1c-99da-4777-bdbf-9f6ef9c341d7 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:42.754859 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 454/893] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:42.756010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef63cc-fed1-41c9-9666-236fe6683273 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:42.844952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fbd1c-99da-4777-bdbf-9f6ef9c341d7 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:42.861780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:42.876187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef63cc-fed1-41c9-9666-236fe6683273 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:42.888684 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b04b8-5036-4cb8-a4bf-fe0c09257367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:42.889321 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b04b8-5036-4cb8-a4bf-fe0c09257367 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.889603 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b04b8-5036-4cb8-a4bf-fe0c09257367 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.889872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b04b8-5036-4cb8-a4bf-fe0c09257367 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.890512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b04b8-5036-4cb8-a4bf-fe0c09257367 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:42.891874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b04b8-5036-4cb8-a4bf-fe0c09257367 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:42.900562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:42.908491 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 440/894] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:19:42 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:42.914311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b47b9-e38b-4082-938b-d83bf77f085e None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.914600 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b47b9-e38b-4082-938b-d83bf77f085e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.914911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b47b9-e38b-4082-938b-d83bf77f085e None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.916948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 441/895] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:42.923868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044ef35-a1b8-4eb2-b4f4-6429475019e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:42.928935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044ef35-a1b8-4eb2-b4f4-6429475019e8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.928935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044ef35-a1b8-4eb2-b4f4-6429475019e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.928935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044ef35-a1b8-4eb2-b4f4-6429475019e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.930912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147b3d5-6bc0-4dcb-9424-5095b2833551 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:42.931802 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147b3d5-6bc0-4dcb-9424-5095b2833551 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:42.932322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147b3d5-6bc0-4dcb-9424-5095b2833551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:42.960930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-a147b3d5-6bc0-4dcb-9424-5095b2833551 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:42.963164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a147b3d5-6bc0-4dcb-9424-5095b2833551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:43.024918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044ef35-a1b8-4eb2-b4f4-6429475019e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:43.026802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044ef35-a1b8-4eb2-b4f4-6429475019e8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:43.049000 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 442/896] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:43.049777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147b3d5-6bc0-4dcb-9424-5095b2833551 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:43.155243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 455/897] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:42 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:19:43.162636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044ef35-a1b8-4eb2-b4f4-6429475019e8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:43.167828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147b3d5-6bc0-4dcb-9424-5095b2833551 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:43.171030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:43.187824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:43.214831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb654aa-9553-4b5d-9b91-c298d4241e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:43.215519 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb654aa-9553-4b5d-9b91-c298d4241e24 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:43.215797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb654aa-9553-4b5d-9b91-c298d4241e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:43.216066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb654aa-9553-4b5d-9b91-c298d4241e24 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:43.216646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb654aa-9553-4b5d-9b91-c298d4241e24 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:43.218442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb654aa-9553-4b5d-9b91-c298d4241e24 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-6eb654aa-9553-4b5d-9b91-c298d4241e24 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:43.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:43.231113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:43.231954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 443/898] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:19:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:43.237500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb654aa-9553-4b5d-9b91-c298d4241e24 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:43.302894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044ef35-a1b8-4eb2-b4f4-6429475019e8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:43.346828 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176314c4-1049-42c9-9e97-5cbee0cf82e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:43.347460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176314c4-1049-42c9-9e97-5cbee0cf82e9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:43.347781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176314c4-1049-42c9-9e97-5cbee0cf82e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:43.348069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176314c4-1049-42c9-9e97-5cbee0cf82e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:43.355302 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb654aa-9553-4b5d-9b91-c298d4241e24 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:43.358319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176314c4-1049-42c9-9e97-5cbee0cf82e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:43.359762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176314c4-1049-42c9-9e97-5cbee0cf82e9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-176314c4-1049-42c9-9e97-5cbee0cf82e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:43.375922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:43.376920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:43.378583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:43.378583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:43.378583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:43.378583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 456/899] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:43.378583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176314c4-1049-42c9-9e97-5cbee0cf82e9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:43.397322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e952393-9af6-4602-8e4b-5861043f6975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:43.397938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e952393-9af6-4602-8e4b-5861043f6975 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:43.398230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e952393-9af6-4602-8e4b-5861043f6975 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:43.398500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e952393-9af6-4602-8e4b-5861043f6975 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:43.399129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e952393-9af6-4602-8e4b-5861043f6975 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:43.399424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e952393-9af6-4602-8e4b-5861043f6975 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:43.400789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e952393-9af6-4602-8e4b-5861043f6975 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:43.405712 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 444/900] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:19:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:43.406438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e952393-9af6-4602-8e4b-5861043f6975 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:43.440217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:43.460596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:43.496852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176314c4-1049-42c9-9e97-5cbee0cf82e9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:43.519734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e952393-9af6-4602-8e4b-5861043f6975 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:43.531924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa9e58-dbfe-4fb3-b5fb-5692f52dd6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:43.532985 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa9e58-dbfe-4fb3-b5fb-5692f52dd6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:43.533449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa9e58-dbfe-4fb3-b5fb-5692f52dd6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:43.534228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa9e58-dbfe-4fb3-b5fb-5692f52dd6e7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:43.535278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa9e58-dbfe-4fb3-b5fb-5692f52dd6e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:43.537010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa9e58-dbfe-4fb3-b5fb-5692f52dd6e7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-5eaa9e58-dbfe-4fb3-b5fb-5692f52dd6e7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:43.548318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:43.549339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 16:19:43.550397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:43.553533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 457/901] 10.222.0.55 () {56 vars in 1132 bytes} [Fri Aug 19 16:19:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:43.554477 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa9e58-dbfe-4fb3-b5fb-5692f52dd6e7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:43.560252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a48fb5-41a5-4afe-8a94-77207c0f71aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:43.560899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a48fb5-41a5-4afe-8a94-77207c0f71aa None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:43.561181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a48fb5-41a5-4afe-8a94-77207c0f71aa None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:43.561453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a48fb5-41a5-4afe-8a94-77207c0f71aa None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:43.562097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a48fb5-41a5-4afe-8a94-77207c0f71aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:43.562401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a48fb5-41a5-4afe-8a94-77207c0f71aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:43.563768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a48fb5-41a5-4afe-8a94-77207c0f71aa None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:43.577241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 445/902] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:43.578053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a48fb5-41a5-4afe-8a94-77207c0f71aa None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:43.670490 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa9e58-dbfe-4fb3-b5fb-5692f52dd6e7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:43.683975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:43.693462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a48fb5-41a5-4afe-8a94-77207c0f71aa None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:43.705021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:43.711031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e7bdf-d72c-40c0-80a2-dc63434f1263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:43.711788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e7bdf-d72c-40c0-80a2-dc63434f1263 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:43.712111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e7bdf-d72c-40c0-80a2-dc63434f1263 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:43.712397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e7bdf-d72c-40c0-80a2-dc63434f1263 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:43.713151 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e7bdf-d72c-40c0-80a2-dc63434f1263 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:43.713458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e7bdf-d72c-40c0-80a2-dc63434f1263 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:43.714937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e7bdf-d72c-40c0-80a2-dc63434f1263 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:43.726255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 458/903] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:19:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:43.727390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e7bdf-d72c-40c0-80a2-dc63434f1263 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:43.728919 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec421b0-15da-477a-87a2-6c2622eb198c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:43.729528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec421b0-15da-477a-87a2-6c2622eb198c None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:43.729810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec421b0-15da-477a-87a2-6c2622eb198c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:43.730086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec421b0-15da-477a-87a2-6c2622eb198c None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:43.730710 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec421b0-15da-477a-87a2-6c2622eb198c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:43.731009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec421b0-15da-477a-87a2-6c2622eb198c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:43.732364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec421b0-15da-477a-87a2-6c2622eb198c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:43.741060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 446/904] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Aug 19 16:19:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:43.742645 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec421b0-15da-477a-87a2-6c2622eb198c None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:43.835389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e7bdf-d72c-40c0-80a2-dc63434f1263 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:43.851500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec421b0-15da-477a-87a2-6c2622eb198c None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:43.870687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe0ace-9e75-4316-997f-1071233719a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:43.871319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe0ace-9e75-4316-997f-1071233719a3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:43.871597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe0ace-9e75-4316-997f-1071233719a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:43.872089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe0ace-9e75-4316-997f-1071233719a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:43.878693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe0ace-9e75-4316-997f-1071233719a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:43.880648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe0ace-9e75-4316-997f-1071233719a3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:43.890105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56ab07-262f-412b-834d-75a897018a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:43.890712 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56ab07-262f-412b-834d-75a897018a9e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:43.890993 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56ab07-262f-412b-834d-75a897018a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:43.891254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56ab07-262f-412b-834d-75a897018a9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-f7fe0ace-9e75-4316-997f-1071233719a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:43.891710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:43.892563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:43.893759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:43.893759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:43.893759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:43.893759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:43.893759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:43.893759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:43.893759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:43.893759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:43.894173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56ab07-262f-412b-834d-75a897018a9e None admin] RBAC: Adding query filter params (interface=public, service_id=c099ce6c20314366b798907acaf1ea45, region_id=RegionOne) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:43.894173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56ab07-262f-412b-834d-75a897018a9e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:43.894310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 459/905] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:43.894310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe0ace-9e75-4316-997f-1071233719a3 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:43.898331 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56ab07-262f-412b-834d-75a897018a9e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:43.913214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 447/906] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Aug 19 16:19:43 2022] GET /identity/v3/endpoints?service_id=c099ce6c20314366b798907acaf1ea45&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:43.913214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56ab07-262f-412b-834d-75a897018a9e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:43.934519 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:43.965014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:44.013232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe0ace-9e75-4316-997f-1071233719a3 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:44.017818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56ab07-262f-412b-834d-75a897018a9e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:44.050129 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878fadd-5505-4c40-a1d6-addc91346682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:44.050769 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878fadd-5505-4c40-a1d6-addc91346682 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:44.051046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878fadd-5505-4c40-a1d6-addc91346682 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:44.051319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878fadd-5505-4c40-a1d6-addc91346682 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:44.051984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878fadd-5505-4c40-a1d6-addc91346682 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:44.052237 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878fadd-5505-4c40-a1d6-addc91346682 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:44.053636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878fadd-5505-4c40-a1d6-addc91346682 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:44.053915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc908b-e4ac-48d0-bef0-fcc9b8000da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:44.054519 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc908b-e4ac-48d0-bef0-fcc9b8000da0 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:44.054808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc908b-e4ac-48d0-bef0-fcc9b8000da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:44.055062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc908b-e4ac-48d0-bef0-fcc9b8000da0 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:44.055741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc908b-e4ac-48d0-bef0-fcc9b8000da0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:44.056039 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc908b-e4ac-48d0-bef0-fcc9b8000da0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:44.057495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc908b-e4ac-48d0-bef0-fcc9b8000da0 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:44.063764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 460/907] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Aug 19 16:19:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:44.063764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878fadd-5505-4c40-a1d6-addc91346682 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:44.075734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 448/908] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:44.076641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc908b-e4ac-48d0-bef0-fcc9b8000da0 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:44.164416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:44.171577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878fadd-5505-4c40-a1d6-addc91346682 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:44.191696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc908b-e4ac-48d0-bef0-fcc9b8000da0 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:44.192698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:44.207748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae097a-c095-4b34-a897-c0305c47216b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:44.208351 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae097a-c095-4b34-a897-c0305c47216b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:44.208628 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae097a-c095-4b34-a897-c0305c47216b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:44.208927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae097a-c095-4b34-a897-c0305c47216b None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:44.209523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae097a-c095-4b34-a897-c0305c47216b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:44.210877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae097a-c095-4b34-a897-c0305c47216b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:44.224242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 461/909] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Aug 19 16:19:44 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:19:44.224242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae097a-c095-4b34-a897-c0305c47216b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:44.246960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803426b-8670-4091-8999-bac3f9d1aa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:44.247591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803426b-8670-4091-8999-bac3f9d1aa89 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:44.248135 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803426b-8670-4091-8999-bac3f9d1aa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:44.248413 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803426b-8670-4091-8999-bac3f9d1aa89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:44.249855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803426b-8670-4091-8999-bac3f9d1aa89 None admin] RBAC: Adding query filter params (interface=public, service_id=c8822487670e4a978cc8ea74ffddf169, region_id=RegionOne) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:44.250284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803426b-8670-4091-8999-bac3f9d1aa89 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:44.252489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803426b-8670-4091-8999-bac3f9d1aa89 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:44.271023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 449/910] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Aug 19 16:19:44 2022] GET /identity/v3/endpoints?service_id=c8822487670e4a978cc8ea74ffddf169&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:44.272365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803426b-8670-4091-8999-bac3f9d1aa89 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:44.338258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae097a-c095-4b34-a897-c0305c47216b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:44.387601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04932fd-43e6-4317-a363-c47cf1d4888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:44.388219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04932fd-43e6-4317-a363-c47cf1d4888e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:44.388779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04932fd-43e6-4317-a363-c47cf1d4888e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:44.388955 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04932fd-43e6-4317-a363-c47cf1d4888e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:44.389679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04932fd-43e6-4317-a363-c47cf1d4888e None admin] RBAC: Adding query filter params (role.id=2dc5eb239bdc4aa6be297199d70bb634, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=88694a50b9f04487a521ba6df110d976) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:44.389985 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04932fd-43e6-4317-a363-c47cf1d4888e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:44.391412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04932fd-43e6-4317-a363-c47cf1d4888e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:44.399282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 462/911] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:44 2022] GET /identity/v3/role_assignments?user.id=88694a50b9f04487a521ba6df110d976&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=2dc5eb239bdc4aa6be297199d70bb634 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:44.418696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803426b-8670-4091-8999-bac3f9d1aa89 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:44.461000 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e87b70-bfff-428f-8d22-d7a03d717e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:44.461537 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e87b70-bfff-428f-8d22-d7a03d717e35 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:44.461813 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e87b70-bfff-428f-8d22-d7a03d717e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:44.462084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e87b70-bfff-428f-8d22-d7a03d717e35 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:44.462697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e87b70-bfff-428f-8d22-d7a03d717e35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:44.464110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e87b70-bfff-428f-8d22-d7a03d717e35 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:44.471902 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 450/912] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Aug 19 16:19:44 2022] GET /identity/v3/services => generated 1497 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:19:44.923305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11bfab-187b-464b-bbf9-1e01bda91b83 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:44.923826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11bfab-187b-464b-bbf9-1e01bda91b83 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:44.923826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11bfab-187b-464b-bbf9-1e01bda91b83 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:44.925459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 463/913] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:44.930936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa519578-67a3-4def-8dc3-019b4c54257c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:44.931350 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa519578-67a3-4def-8dc3-019b4c54257c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:44.931521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa519578-67a3-4def-8dc3-019b4c54257c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:44.949952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-aa519578-67a3-4def-8dc3-019b4c54257c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:44.951144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-aa519578-67a3-4def-8dc3-019b4c54257c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:44.953325 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:45.024557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 451/914] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:45.303624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11bfab-187b-464b-bbf9-1e01bda91b83 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:45.359881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:45.419578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11bfab-187b-464b-bbf9-1e01bda91b83 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:45.455583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a37d90-8111-4fd3-8179-00edad3932f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:45.456222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a37d90-8111-4fd3-8179-00edad3932f1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:45.456497 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a37d90-8111-4fd3-8179-00edad3932f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:45.456813 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a37d90-8111-4fd3-8179-00edad3932f1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:45.457409 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a37d90-8111-4fd3-8179-00edad3932f1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:45.458768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a37d90-8111-4fd3-8179-00edad3932f1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-09a37d90-8111-4fd3-8179-00edad3932f1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:45.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:45.469788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:45.470890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 464/915] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Aug 19 16:19:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:45.474729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa519578-67a3-4def-8dc3-019b4c54257c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:45.588499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa519578-67a3-4def-8dc3-019b4c54257c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:45.624905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b429da-33b7-4416-8308-15b9ed552c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:45.625518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b429da-33b7-4416-8308-15b9ed552c71 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:45.625798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b429da-33b7-4416-8308-15b9ed552c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:45.626071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b429da-33b7-4416-8308-15b9ed552c71 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:45.626707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b429da-33b7-4416-8308-15b9ed552c71 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:45.627010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b429da-33b7-4416-8308-15b9ed552c71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:45.628372 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b429da-33b7-4416-8308-15b9ed552c71 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:45.633725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 452/916] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Aug 19 16:19:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:45.641572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a37d90-8111-4fd3-8179-00edad3932f1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:45.748169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a37d90-8111-4fd3-8179-00edad3932f1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:45.761914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:45.784605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e357c-8e35-4c23-91d5-7196dc832600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:45.785847 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e357c-8e35-4c23-91d5-7196dc832600 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:45.786121 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e357c-8e35-4c23-91d5-7196dc832600 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:45.786390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e357c-8e35-4c23-91d5-7196dc832600 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:45.787016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e357c-8e35-4c23-91d5-7196dc832600 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:45.787313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e357c-8e35-4c23-91d5-7196dc832600 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:45.788669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e357c-8e35-4c23-91d5-7196dc832600 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:45.793946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 465/917] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Aug 19 16:19:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:45.802143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b429da-33b7-4416-8308-15b9ed552c71 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:45.805336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:45.917486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b429da-33b7-4416-8308-15b9ed552c71 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:45.953878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122052ff-dfb1-4eb8-b92a-ea7ad936cf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:45.955218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122052ff-dfb1-4eb8-b92a-ea7ad936cf36 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:45.955717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122052ff-dfb1-4eb8-b92a-ea7ad936cf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:45.956174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122052ff-dfb1-4eb8-b92a-ea7ad936cf36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:45.957663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122052ff-dfb1-4eb8-b92a-ea7ad936cf36 None admin] RBAC: Adding query filter params (interface=public, service_id=a0432903fce543b6925f3b0f13805706, region_id=RegionOne) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:45.958176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122052ff-dfb1-4eb8-b92a-ea7ad936cf36 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:45.959718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122052ff-dfb1-4eb8-b92a-ea7ad936cf36 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:45.965636 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 453/918] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Aug 19 16:19:45 2022] GET /identity/v3/endpoints?service_id=a0432903fce543b6925f3b0f13805706&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:45.972775 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e357c-8e35-4c23-91d5-7196dc832600 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:46.079005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e357c-8e35-4c23-91d5-7196dc832600 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:46.114182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ca86d-3780-46bc-ad4f-c1940c8ef205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:46.114790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ca86d-3780-46bc-ad4f-c1940c8ef205 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:46.115066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ca86d-3780-46bc-ad4f-c1940c8ef205 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:46.115334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ca86d-3780-46bc-ad4f-c1940c8ef205 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:46.115933 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ca86d-3780-46bc-ad4f-c1940c8ef205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:46.117341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ca86d-3780-46bc-ad4f-c1940c8ef205 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:46.123927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 466/919] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Aug 19 16:19:45 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:19:46.249922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2f50e-d69d-421b-a4bc-988f56faa4ce None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:46.250600 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2f50e-d69d-421b-a4bc-988f56faa4ce None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:46.251105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2f50e-d69d-421b-a4bc-988f56faa4ce None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:46.253018 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 454/920] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:46.259514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6d3c8-ecf1-44fc-85e2-9714328eddc9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:46.259809 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6d3c8-ecf1-44fc-85e2-9714328eddc9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:46.260076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6d3c8-ecf1-44fc-85e2-9714328eddc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:46.278914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-28f6d3c8-ecf1-44fc-85e2-9714328eddc9 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:46.280099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-28f6d3c8-ecf1-44fc-85e2-9714328eddc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:46.317282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:46.350684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 467/921] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:46.390696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9425268-7029-4903-a544-dd15f98c6d25 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:46.391027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9425268-7029-4903-a544-dd15f98c6d25 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:46.391237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9425268-7029-4903-a544-dd15f98c6d25 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:46.392921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 455/922] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:46.398135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ae792-641b-417b-a4e6-9e92815cf7c3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:46.398424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ae792-641b-417b-a4e6-9e92815cf7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:46.398695 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ae792-641b-417b-a4e6-9e92815cf7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:46.416942 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c92ae792-641b-417b-a4e6-9e92815cf7c3 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:46.418125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c92ae792-641b-417b-a4e6-9e92815cf7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:46.440301 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcc327-e917-4963-bdb1-3d0f9d1903d0 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:46.440626 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcc327-e917-4963-bdb1-3d0f9d1903d0 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:46.440882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcc327-e917-4963-bdb1-3d0f9d1903d0 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:46.443317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 456/923] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:46.448466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cda920-4a55-493a-9605-00abc6061447 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:46.448811 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cda920-4a55-493a-9605-00abc6061447 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:46.449042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cda920-4a55-493a-9605-00abc6061447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:46.469919 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d5cda920-4a55-493a-9605-00abc6061447 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:46.471409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d5cda920-4a55-493a-9605-00abc6061447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:46.491846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 468/924] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:46.546423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:46.555888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 457/925] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:46.639022 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ae792-641b-417b-a4e6-9e92815cf7c3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:46.749207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:46.752967 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ae792-641b-417b-a4e6-9e92815cf7c3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:46.771320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cda920-4a55-493a-9605-00abc6061447 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:46.790442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef02725-a6e2-48ba-99d7-3fd6492d59af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:46.791510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef02725-a6e2-48ba-99d7-3fd6492d59af None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:46.791991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef02725-a6e2-48ba-99d7-3fd6492d59af None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:46.792463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef02725-a6e2-48ba-99d7-3fd6492d59af None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:46.793279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef02725-a6e2-48ba-99d7-3fd6492d59af None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:46.794879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef02725-a6e2-48ba-99d7-3fd6492d59af None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-0ef02725-a6e2-48ba-99d7-3fd6492d59af None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:46.804785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:46.805720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 16:19:46.806534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:46.809291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 469/926] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:19:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:46.815813 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef02725-a6e2-48ba-99d7-3fd6492d59af None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:46.887674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cda920-4a55-493a-9605-00abc6061447 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:46.923563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198052b6-6d4b-4e8a-a75a-d3b864987c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:46.924188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198052b6-6d4b-4e8a-a75a-d3b864987c1f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:46.924462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198052b6-6d4b-4e8a-a75a-d3b864987c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:46.924728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198052b6-6d4b-4e8a-a75a-d3b864987c1f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:46.935288 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198052b6-6d4b-4e8a-a75a-d3b864987c1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:46.936686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef02725-a6e2-48ba-99d7-3fd6492d59af None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-198052b6-6d4b-4e8a-a75a-d3b864987c1f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:46.939897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:46.940997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:46.942270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:46.942270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:46.942270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:46.942270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:46.942270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:46.942270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:46.942270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:46.942270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:46.942270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:46.942270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:46.942270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 16:19:46.942270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:46.946960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 458/927] 10.222.0.55 () {56 vars in 1120 bytes} [Fri Aug 19 16:19:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:46.948240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198052b6-6d4b-4e8a-a75a-d3b864987c1f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:46.974322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e19e1-cc9c-45ca-a612-3b404abd361d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:46.974924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e19e1-cc9c-45ca-a612-3b404abd361d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:46.975202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e19e1-cc9c-45ca-a612-3b404abd361d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:46.975469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e19e1-cc9c-45ca-a612-3b404abd361d None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:46.976087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e19e1-cc9c-45ca-a612-3b404abd361d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:46.976382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e19e1-cc9c-45ca-a612-3b404abd361d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:46.978034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e19e1-cc9c-45ca-a612-3b404abd361d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:46.982622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 470/928] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:19:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:46.983381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e19e1-cc9c-45ca-a612-3b404abd361d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:47.011037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:47.024649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:47.068275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198052b6-6d4b-4e8a-a75a-d3b864987c1f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:47.091990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e19e1-cc9c-45ca-a612-3b404abd361d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:47.104970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e9a5d-df98-41f3-9b8a-39db869b93df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:47.105609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e9a5d-df98-41f3-9b8a-39db869b93df None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:47.105890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e9a5d-df98-41f3-9b8a-39db869b93df None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:47.106162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e9a5d-df98-41f3-9b8a-39db869b93df None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:47.106755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e9a5d-df98-41f3-9b8a-39db869b93df None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:47.108123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e9a5d-df98-41f3-9b8a-39db869b93df None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-3d9e9a5d-df98-41f3-9b8a-39db869b93df None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:47.116886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:47.117821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:47.118690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 459/929] 10.222.0.55 () {56 vars in 1132 bytes} [Fri Aug 19 16:19:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:47.119461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e9a5d-df98-41f3-9b8a-39db869b93df None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:47.133529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61c9da-fde5-4875-aa71-70930031849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:47.134150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61c9da-fde5-4875-aa71-70930031849a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:47.134427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61c9da-fde5-4875-aa71-70930031849a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:47.134702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61c9da-fde5-4875-aa71-70930031849a None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:47.135371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61c9da-fde5-4875-aa71-70930031849a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:47.135711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61c9da-fde5-4875-aa71-70930031849a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:47.137151 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61c9da-fde5-4875-aa71-70930031849a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:47.145938 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 471/930] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Aug 19 16:19:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:47.146757 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61c9da-fde5-4875-aa71-70930031849a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:47.232571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e9a5d-df98-41f3-9b8a-39db869b93df None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:47.248355 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:47.258041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:47.261682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61c9da-fde5-4875-aa71-70930031849a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:47.274227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666878d-0d9c-4b2b-93a3-a906c6f90d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:47.274821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666878d-0d9c-4b2b-93a3-a906c6f90d46 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:47.275110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666878d-0d9c-4b2b-93a3-a906c6f90d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:47.275392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666878d-0d9c-4b2b-93a3-a906c6f90d46 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:47.276016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666878d-0d9c-4b2b-93a3-a906c6f90d46 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:47.276323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666878d-0d9c-4b2b-93a3-a906c6f90d46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:47.277723 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666878d-0d9c-4b2b-93a3-a906c6f90d46 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:47.282583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 460/931] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Aug 19 16:19:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:47.283355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666878d-0d9c-4b2b-93a3-a906c6f90d46 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:47.303920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd912bf7-8de4-472c-8eb2-39051463bd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:47.304543 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd912bf7-8de4-472c-8eb2-39051463bd76 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:47.304860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd912bf7-8de4-472c-8eb2-39051463bd76 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:47.305146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd912bf7-8de4-472c-8eb2-39051463bd76 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:47.305780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd912bf7-8de4-472c-8eb2-39051463bd76 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:47.306087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd912bf7-8de4-472c-8eb2-39051463bd76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:47.307448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd912bf7-8de4-472c-8eb2-39051463bd76 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:47.316967 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 472/932] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Aug 19 16:19:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:47.316967 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd912bf7-8de4-472c-8eb2-39051463bd76 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:47.396651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666878d-0d9c-4b2b-93a3-a906c6f90d46 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:47.431568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd912bf7-8de4-472c-8eb2-39051463bd76 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:47.433383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20643c-2c59-4961-9b12-c558a610c2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:47.434287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20643c-2c59-4961-9b12-c558a610c2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:47.434564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20643c-2c59-4961-9b12-c558a610c2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:47.434832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20643c-2c59-4961-9b12-c558a610c2f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:47.446434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20643c-2c59-4961-9b12-c558a610c2f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:47.448006 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20643c-2c59-4961-9b12-c558a610c2f7 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-9c20643c-2c59-4961-9b12-c558a610c2f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:47.463392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:47.464323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:47.465306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:47.465306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:47.465306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:47.465306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 461/933] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:47.465306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20643c-2c59-4961-9b12-c558a610c2f7 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:47.468379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544058c6-fa02-4951-9b6f-f4a7e220e9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:47.469208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544058c6-fa02-4951-9b6f-f4a7e220e9bb None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:47.469371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544058c6-fa02-4951-9b6f-f4a7e220e9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:47.469648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544058c6-fa02-4951-9b6f-f4a7e220e9bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:47.470310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544058c6-fa02-4951-9b6f-f4a7e220e9bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:47.471685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544058c6-fa02-4951-9b6f-f4a7e220e9bb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:47.494321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 473/934] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:19:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:47.495140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544058c6-fa02-4951-9b6f-f4a7e220e9bb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:47.517421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:47.520563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:47.616004 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20643c-2c59-4961-9b12-c558a610c2f7 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:47.646213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544058c6-fa02-4951-9b6f-f4a7e220e9bb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:47.666484 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342d0f7-3893-4a6f-aad8-8d96bc97c3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:47.667609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342d0f7-3893-4a6f-aad8-8d96bc97c3dd None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:47.668041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342d0f7-3893-4a6f-aad8-8d96bc97c3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:47.668454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342d0f7-3893-4a6f-aad8-8d96bc97c3dd None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:47.669408 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342d0f7-3893-4a6f-aad8-8d96bc97c3dd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:47.669870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342d0f7-3893-4a6f-aad8-8d96bc97c3dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:47.671692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342d0f7-3893-4a6f-aad8-8d96bc97c3dd None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:47.678807 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 462/935] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Aug 19 16:19:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:47.685104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342d0f7-3893-4a6f-aad8-8d96bc97c3dd None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:47.703374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbcd7f-5df7-4a07-9f93-3ecf15d11ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:47.703374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbcd7f-5df7-4a07-9f93-3ecf15d11ebe None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:47.703667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbcd7f-5df7-4a07-9f93-3ecf15d11ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:47.703916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbcd7f-5df7-4a07-9f93-3ecf15d11ebe None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:47.705153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbcd7f-5df7-4a07-9f93-3ecf15d11ebe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:47.705153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbcd7f-5df7-4a07-9f93-3ecf15d11ebe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:47.706421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbcd7f-5df7-4a07-9f93-3ecf15d11ebe None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:47.731148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 474/936] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:47.737807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbcd7f-5df7-4a07-9f93-3ecf15d11ebe None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:47.822730 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:47.825429 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:47.827620 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342d0f7-3893-4a6f-aad8-8d96bc97c3dd None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:47.871116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6a81d-3207-49ec-9305-0a2138d7bdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:47.872137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6a81d-3207-49ec-9305-0a2138d7bdb9 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:47.872678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6a81d-3207-49ec-9305-0a2138d7bdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:47.873126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbcd7f-5df7-4a07-9f93-3ecf15d11ebe None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:47.873347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6a81d-3207-49ec-9305-0a2138d7bdb9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:47.874182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6a81d-3207-49ec-9305-0a2138d7bdb9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:47.875997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6a81d-3207-49ec-9305-0a2138d7bdb9 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:47.904436 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 463/937] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:19:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:47.933325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b99a00-578b-483a-9d2f-8455e89ce95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:47.934236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b99a00-578b-483a-9d2f-8455e89ce95b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:47.934686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b99a00-578b-483a-9d2f-8455e89ce95b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:47.935108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b99a00-578b-483a-9d2f-8455e89ce95b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:47.944897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b99a00-578b-483a-9d2f-8455e89ce95b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:47.946747 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b99a00-578b-483a-9d2f-8455e89ce95b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-f9b99a00-578b-483a-9d2f-8455e89ce95b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:47.963118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:47.964322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:47.965555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:47.965555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:47.965555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:47.965555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:47.965555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:47.965555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:47.965555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:47.965555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:47.965555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 16:19:47.965555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:47.968060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 475/938] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Aug 19 16:19:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:47.974242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6a81d-3207-49ec-9305-0a2138d7bdb9 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:48.105182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6a81d-3207-49ec-9305-0a2138d7bdb9 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:48.148317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc589819-6199-490f-93ec-4bf73e9daea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:48.148976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc589819-6199-490f-93ec-4bf73e9daea1 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:48.149245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc589819-6199-490f-93ec-4bf73e9daea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:48.149513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc589819-6199-490f-93ec-4bf73e9daea1 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:48.150243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc589819-6199-490f-93ec-4bf73e9daea1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:48.150544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc589819-6199-490f-93ec-4bf73e9daea1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:48.151971 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc589819-6199-490f-93ec-4bf73e9daea1 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:48.157444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3ca4e-bcfc-4246-b6fb-e5fadd921535 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:48.157920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3ca4e-bcfc-4246-b6fb-e5fadd921535 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:48.157998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3ca4e-bcfc-4246-b6fb-e5fadd921535 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:48.159541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 476/939] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:48.167570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cfe78-adca-43ca-a3ba-72342c9f2aa7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:48.168072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cfe78-adca-43ca-a3ba-72342c9f2aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:48.168551 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cfe78-adca-43ca-a3ba-72342c9f2aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:48.173732 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 464/940] 10.222.0.55 () {56 vars in 1149 bytes} [Fri Aug 19 16:19:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:48.179978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc589819-6199-490f-93ec-4bf73e9daea1 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:48.194969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-995cfe78-adca-43ca-a3ba-72342c9f2aa7 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:48.196109 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-995cfe78-adca-43ca-a3ba-72342c9f2aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:48.202591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:48.308938 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 477/941] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:48.335333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc589819-6199-490f-93ec-4bf73e9daea1 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:48.385843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb06320-a426-4a34-b7ac-98de103caff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:48.386748 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb06320-a426-4a34-b7ac-98de103caff9 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:48.387240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb06320-a426-4a34-b7ac-98de103caff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:48.387781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb06320-a426-4a34-b7ac-98de103caff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:48.403232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb06320-a426-4a34-b7ac-98de103caff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:48.405232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb06320-a426-4a34-b7ac-98de103caff9 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-adb06320-a426-4a34-b7ac-98de103caff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:48.429206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:48.430879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:48.432499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:48.432499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:48.432499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:48.432499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 465/942] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:48.440159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cfe78-adca-43ca-a3ba-72342c9f2aa7 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:48.513581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:48.585910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cfe78-adca-43ca-a3ba-72342c9f2aa7 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:48.626398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e248694-f9e3-4cd5-8c62-2587b5ab67f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:48.627398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e248694-f9e3-4cd5-8c62-2587b5ab67f0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:48.627899 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e248694-f9e3-4cd5-8c62-2587b5ab67f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:48.628415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e248694-f9e3-4cd5-8c62-2587b5ab67f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:48.630218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e248694-f9e3-4cd5-8c62-2587b5ab67f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:48.630775 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e248694-f9e3-4cd5-8c62-2587b5ab67f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:48.633425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e248694-f9e3-4cd5-8c62-2587b5ab67f0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:48.639852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb06320-a426-4a34-b7ac-98de103caff9 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:48.654044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 478/943] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:48.657288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e248694-f9e3-4cd5-8c62-2587b5ab67f0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:48.722109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:48.787347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e248694-f9e3-4cd5-8c62-2587b5ab67f0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:48.797487 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb06320-a426-4a34-b7ac-98de103caff9 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:48.804650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:48.833366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4937a-a28a-4284-8cac-1879a1b74ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:48.834553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4937a-a28a-4284-8cac-1879a1b74ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:48.835261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4937a-a28a-4284-8cac-1879a1b74ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:48.835939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4937a-a28a-4284-8cac-1879a1b74ba0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:48.840271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f9ad6-37c2-4a91-be20-a8f12f9aabdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:48.841066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f9ad6-37c2-4a91-be20-a8f12f9aabdd None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:48.841496 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f9ad6-37c2-4a91-be20-a8f12f9aabdd None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:48.841876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f9ad6-37c2-4a91-be20-a8f12f9aabdd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:48.842558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f9ad6-37c2-4a91-be20-a8f12f9aabdd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:48.844193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4937a-a28a-4284-8cac-1879a1b74ba0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:48.844393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f9ad6-37c2-4a91-be20-a8f12f9aabdd None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:48.845934 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4937a-a28a-4284-8cac-1879a1b74ba0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-7f9f9ad6-37c2-4a91-be20-a8f12f9aabdd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:48.855385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:48.856675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:48.857699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:48.858499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 466/944] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Aug 19 16:19:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:48.865499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f9ad6-37c2-4a91-be20-a8f12f9aabdd None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-37a4937a-a28a-4284-8cac-1879a1b74ba0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:48.866665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:48.867512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:48.869883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:48.869883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:48.869883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:48.869883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:48.869883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:48.869883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:48.869883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:48.869883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:48.869883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 479/945] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:48.876681 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4937a-a28a-4284-8cac-1879a1b74ba0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:49.006457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f9ad6-37c2-4a91-be20-a8f12f9aabdd None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:49.012179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4937a-a28a-4284-8cac-1879a1b74ba0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:49.042183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:49.048052 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba9122-3af3-44b8-b9d7-a052620caac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:49.048950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9122-3af3-44b8-b9d7-a052620caac0 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:49.049536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9122-3af3-44b8-b9d7-a052620caac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:49.050023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba9122-3af3-44b8-b9d7-a052620caac0 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:49.051817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba9122-3af3-44b8-b9d7-a052620caac0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:49.052172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba9122-3af3-44b8-b9d7-a052620caac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:49.053693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba9122-3af3-44b8-b9d7-a052620caac0 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:49.063235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 467/946] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Aug 19 16:19:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:49.064338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba9122-3af3-44b8-b9d7-a052620caac0 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:49.065595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384b383-a6f9-44a0-91dc-8df108ef7234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:49.066546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384b383-a6f9-44a0-91dc-8df108ef7234 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:49.067456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384b383-a6f9-44a0-91dc-8df108ef7234 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:49.067937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384b383-a6f9-44a0-91dc-8df108ef7234 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:49.069784 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384b383-a6f9-44a0-91dc-8df108ef7234 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:49.070415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384b383-a6f9-44a0-91dc-8df108ef7234 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:49.072073 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384b383-a6f9-44a0-91dc-8df108ef7234 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:49.091240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 480/947] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:49.100526 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384b383-a6f9-44a0-91dc-8df108ef7234 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:49.140232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:49.178314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba9122-3af3-44b8-b9d7-a052620caac0 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:49.205099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384b383-a6f9-44a0-91dc-8df108ef7234 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:49.213724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92d2aa-fd1b-4b3c-840e-3786d135a9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:49.214361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92d2aa-fd1b-4b3c-840e-3786d135a9ec None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:49.214636 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92d2aa-fd1b-4b3c-840e-3786d135a9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:49.214906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92d2aa-fd1b-4b3c-840e-3786d135a9ec None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:49.215536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92d2aa-fd1b-4b3c-840e-3786d135a9ec None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:49.215830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92d2aa-fd1b-4b3c-840e-3786d135a9ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:49.217217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92d2aa-fd1b-4b3c-840e-3786d135a9ec None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:49.228132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 468/948] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Aug 19 16:19:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:49.229011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92d2aa-fd1b-4b3c-840e-3786d135a9ec None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:49.242669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea0bd4-4e05-4d84-a521-b3aa6e99d1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:49.243393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea0bd4-4e05-4d84-a521-b3aa6e99d1a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:49.243673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea0bd4-4e05-4d84-a521-b3aa6e99d1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:49.243993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea0bd4-4e05-4d84-a521-b3aa6e99d1a1 None admin] PATH_INFO: `/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/88694a50b9f04487a521ba6df110d976/roles/2dc5eb239bdc4aa6be297199d70bb634` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:49.246281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea0bd4-4e05-4d84-a521-b3aa6e99d1a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d143cf444f354ec0bf2089dae3d94255, user_id=88694a50b9f04487a521ba6df110d976, role_id=2dc5eb239bdc4aa6be297199d70bb634)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:49.247723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea0bd4-4e05-4d84-a521-b3aa6e99d1a1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:49.264336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 481/949] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:19:49 2022] PUT /identity/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/88694a50b9f04487a521ba6df110d976/roles/2dc5eb239bdc4aa6be297199d70bb634 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:19:49.301501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:49.349964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92d2aa-fd1b-4b3c-840e-3786d135a9ec None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:49.387287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5fe32-3c9c-40b4-a00f-65399a2d039b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:49.387911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5fe32-3c9c-40b4-a00f-65399a2d039b None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:49.388191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5fe32-3c9c-40b4-a00f-65399a2d039b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:49.388465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5fe32-3c9c-40b4-a00f-65399a2d039b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:49.389182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5fe32-3c9c-40b4-a00f-65399a2d039b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:49.390573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5fe32-3c9c-40b4-a00f-65399a2d039b None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:49.416613 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 469/950] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:19:49 2022] POST /identity/v3/endpoints => generated 349 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:49.586271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b75b0-be42-49a4-9804-478c368fdfaf None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:49.586576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b75b0-be42-49a4-9804-478c368fdfaf None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:49.586759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b75b0-be42-49a4-9804-478c368fdfaf None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:49.588350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 482/951] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:49.593701 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b2f8a-1e64-411c-9e3a-2520eaefad71 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:49.593995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b2f8a-1e64-411c-9e3a-2520eaefad71 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:49.594274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b2f8a-1e64-411c-9e3a-2520eaefad71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:49.613426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-464b2f8a-1e64-411c-9e3a-2520eaefad71 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:49.614746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-464b2f8a-1e64-411c-9e3a-2520eaefad71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:49.701594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 470/952] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:49.975791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b75b0-be42-49a4-9804-478c368fdfaf None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:50.014037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4de28-059c-40a1-9f07-716e3dc806f5 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:50.014409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4de28-059c-40a1-9f07-716e3dc806f5 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:50.014592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4de28-059c-40a1-9f07-716e3dc806f5 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:50.016094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 471/953] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:50.021271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf49ccf-6e8d-478e-a953-08970c7c9119 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:50.021555 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf49ccf-6e8d-478e-a953-08970c7c9119 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:50.021825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf49ccf-6e8d-478e-a953-08970c7c9119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:50.040152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-2bf49ccf-6e8d-478e-a953-08970c7c9119 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:50.041465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2bf49ccf-6e8d-478e-a953-08970c7c9119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:50.083529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:50.087836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:50.096983 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b75b0-be42-49a4-9804-478c368fdfaf None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:50.135224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46cdfa6-0d74-4040-9daa-6fb2d5752dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:50.136113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cdfa6-0d74-4040-9daa-6fb2d5752dbb None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:50.136620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cdfa6-0d74-4040-9daa-6fb2d5752dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:50.136848 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 472/954] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:50.137326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cdfa6-0d74-4040-9daa-6fb2d5752dbb None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:50.138380 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46cdfa6-0d74-4040-9daa-6fb2d5752dbb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:50.140256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46cdfa6-0d74-4040-9daa-6fb2d5752dbb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:50.143539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: WARNING keystone.server.flask.common [None req-a46cdfa6-0d74-4040-9daa-6fb2d5752dbb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 19 16:19:50.162930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.sql.core [None req-a46cdfa6-0d74-4040-9daa-6fb2d5752dbb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 19 16:19:50.162930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 19 16:19:50.162930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [parameters: {'id': '85b8580c0d474ed38bed71fb89c51b04', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 19 16:19:50.162930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81551) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a46cdfa6-0d74-4040-9daa-6fb2d5752dbb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 85b8580c0d474ed38bed71fb89c51b04.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 85b8580c0d474ed38bed71fb89c51b04. Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application conn.query(q) Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 16:19:50.171816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result.read() Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 19 16:19:50.172686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application next(self.gen) Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application yield resource Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application next(self.gen) Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.session.rollback() Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.force_reraise() Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise self.value Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application session.commit() Aug 19 16:19:50.173743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._prepare_impl() Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.session.flush() Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._flush(objects) Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application compat.raise_( Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application flush_context.execute() Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 19 16:19:50.174684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rec.execute(self) Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 19 16:19:50.176235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application conn.query(q) Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result.read() Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 16:19:50.178734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [parameters: {'id': '85b8580c0d474ed38bed71fb89c51b04', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:50.181053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:50.183137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:50.187036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:50.187036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 19 16:19:50.187036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 19 16:19:50.187036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:50.187036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:50.187036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 19 16:19:50.187036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 19 16:19:50.187036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 85b8580c0d474ed38bed71fb89c51b04. Aug 19 16:19:50.187036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:50.188620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 483/955] 10.222.0.55 () {60 vars in 1140 bytes} [Fri Aug 19 16:19:49 2022] POST /identity/v3/projects => generated 262 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:50.198691 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf49ccf-6e8d-478e-a953-08970c7c9119 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:50.305219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf49ccf-6e8d-478e-a953-08970c7c9119 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:50.341411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722450dd-57fb-48f4-abea-d9225160efba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:50.342404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722450dd-57fb-48f4-abea-d9225160efba None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:50.342910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722450dd-57fb-48f4-abea-d9225160efba None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:50.343398 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722450dd-57fb-48f4-abea-d9225160efba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:50.351432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722450dd-57fb-48f4-abea-d9225160efba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:50.353441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722450dd-57fb-48f4-abea-d9225160efba None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-722450dd-57fb-48f4-abea-d9225160efba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:50.365990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:50.366953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:50.367885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:50.367885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:50.367885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:50.367885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:50.367885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:50.367885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:50.367885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:50.367885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:50.370540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 473/956] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:50.377317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46cdfa6-0d74-4040-9daa-6fb2d5752dbb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:50.437425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722450dd-57fb-48f4-abea-d9225160efba None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:50.470029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:50.491309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46cdfa6-0d74-4040-9daa-6fb2d5752dbb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:50.529763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b365214-c60a-4aa7-8d0a-81bb508a78b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:50.530771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365214-c60a-4aa7-8d0a-81bb508a78b6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:50.531258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365214-c60a-4aa7-8d0a-81bb508a78b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:50.531781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365214-c60a-4aa7-8d0a-81bb508a78b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:50.532892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b365214-c60a-4aa7-8d0a-81bb508a78b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:50.533408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b365214-c60a-4aa7-8d0a-81bb508a78b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:50.536036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b365214-c60a-4aa7-8d0a-81bb508a78b6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:50.544678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722450dd-57fb-48f4-abea-d9225160efba None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:50.564986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 484/957] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:50.582463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce1341-f756-427f-893b-e0f1a1d7c4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:50.583149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce1341-f756-427f-893b-e0f1a1d7c4cc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:50.583426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce1341-f756-427f-893b-e0f1a1d7c4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:50.583697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce1341-f756-427f-893b-e0f1a1d7c4cc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:50.584303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce1341-f756-427f-893b-e0f1a1d7c4cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:50.585785 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce1341-f756-427f-893b-e0f1a1d7c4cc None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-51ce1341-f756-427f-893b-e0f1a1d7c4cc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:50.595483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:50.596605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 16:19:50.598161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:50.602430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 474/958] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:50.603853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b365214-c60a-4aa7-8d0a-81bb508a78b6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:50.616399 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:50.730244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b365214-c60a-4aa7-8d0a-81bb508a78b6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:50.766635 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97759cf-b12a-4447-8c33-5bd1c4844153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:50.767527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97759cf-b12a-4447-8c33-5bd1c4844153 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:50.767977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97759cf-b12a-4447-8c33-5bd1c4844153 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:50.768413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97759cf-b12a-4447-8c33-5bd1c4844153 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:50.769742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97759cf-b12a-4447-8c33-5bd1c4844153 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:50.770218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97759cf-b12a-4447-8c33-5bd1c4844153 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:50.771746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97759cf-b12a-4447-8c33-5bd1c4844153 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:50.777696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 485/959] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:50.783730 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce1341-f756-427f-893b-e0f1a1d7c4cc None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:50.886501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:50.896394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce1341-f756-427f-893b-e0f1a1d7c4cc None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:50.933446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e57e46-0d6d-4bde-9f5e-1f37d5902cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:50.934311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e57e46-0d6d-4bde-9f5e-1f37d5902cdb None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:50.934761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e57e46-0d6d-4bde-9f5e-1f37d5902cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:50.935179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e57e46-0d6d-4bde-9f5e-1f37d5902cdb None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:50.935966 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e57e46-0d6d-4bde-9f5e-1f37d5902cdb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:50.936432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e57e46-0d6d-4bde-9f5e-1f37d5902cdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:50.938081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e57e46-0d6d-4bde-9f5e-1f37d5902cdb None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:50.943563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 475/960] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:51.209570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5a022-27fa-4316-b694-1c0ec55f063c None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:51.210257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5a022-27fa-4316-b694-1c0ec55f063c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:51.210769 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5a022-27fa-4316-b694-1c0ec55f063c None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:51.212656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 486/961] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:51.217535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16445810-d02e-462c-a122-f9842d2313bb None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:51.217987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16445810-d02e-462c-a122-f9842d2313bb None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:51.218066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16445810-d02e-462c-a122-f9842d2313bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:51.236332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-16445810-d02e-462c-a122-f9842d2313bb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:51.237547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-16445810-d02e-462c-a122-f9842d2313bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:51.305203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 476/962] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:51.433532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018a505-cbf9-43f6-bdcd-70afc64c7a3b None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:51.434078 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018a505-cbf9-43f6-bdcd-70afc64c7a3b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:51.434078 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018a505-cbf9-43f6-bdcd-70afc64c7a3b None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:51.435666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 487/963] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:51.440911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f436e2-be7c-4c85-851b-14530075e2c3 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:51.441908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f436e2-be7c-4c85-851b-14530075e2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:51.442018 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f436e2-be7c-4c85-851b-14530075e2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:51.459946 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-12f436e2-be7c-4c85-851b-14530075e2c3 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:51.461079 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-12f436e2-be7c-4c85-851b-14530075e2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:51.531020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 477/964] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:51.576403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018a505-cbf9-43f6-bdcd-70afc64c7a3b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:51.652716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:51.687072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018a505-cbf9-43f6-bdcd-70afc64c7a3b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:51.722256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea0d9d-ccc4-4851-b8c7-d536a03c1778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:51.722817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea0d9d-ccc4-4851-b8c7-d536a03c1778 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:51.723094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea0d9d-ccc4-4851-b8c7-d536a03c1778 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:51.723366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea0d9d-ccc4-4851-b8c7-d536a03c1778 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:51.733965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea0d9d-ccc4-4851-b8c7-d536a03c1778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c3ea0d9d-ccc4-4851-b8c7-d536a03c1778 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:19:51.739336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:51.740275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:51.741202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:51.741202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:51.741202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:51.741202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:51.741202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:19:51.741202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:19:51.741202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:19:51.741202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:19:51.741202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:19:51.741202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:19:51.741202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 16:19:51.741202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:51.744864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 488/965] 10.222.0.55 () {56 vars in 1120 bytes} [Fri Aug 19 16:19:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:51.746725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f436e2-be7c-4c85-851b-14530075e2c3 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:51.766830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:51.804465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea0d9d-ccc4-4851-b8c7-d536a03c1778 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:51.853394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f436e2-be7c-4c85-851b-14530075e2c3 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:51.901574 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f1d3f-c24f-4be9-8c04-2ff60e3e0597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:51.902435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f1d3f-c24f-4be9-8c04-2ff60e3e0597 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:51.903256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f1d3f-c24f-4be9-8c04-2ff60e3e0597 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:51.903687 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f1d3f-c24f-4be9-8c04-2ff60e3e0597 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:51.904796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f1d3f-c24f-4be9-8c04-2ff60e3e0597 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:51.905288 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f1d3f-c24f-4be9-8c04-2ff60e3e0597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:51.907105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f1d3f-c24f-4be9-8c04-2ff60e3e0597 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:51.918676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 478/966] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Aug 19 16:19:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:51.924829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f1d3f-c24f-4be9-8c04-2ff60e3e0597 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:51.935700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea0d9d-ccc4-4851-b8c7-d536a03c1778 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:51.968963 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:51.977249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826c963-0221-448b-ab2e-c09f7a755f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:51.977968 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826c963-0221-448b-ab2e-c09f7a755f8f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:51.978722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826c963-0221-448b-ab2e-c09f7a755f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:51.979136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826c963-0221-448b-ab2e-c09f7a755f8f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:51.979825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826c963-0221-448b-ab2e-c09f7a755f8f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:51.981341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826c963-0221-448b-ab2e-c09f7a755f8f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c826c963-0221-448b-ab2e-c09f7a755f8f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:51.995016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:51.996106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:51.996982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 489/967] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Aug 19 16:19:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:52.003949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826c963-0221-448b-ab2e-c09f7a755f8f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:52.019500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:52.041970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f1d3f-c24f-4be9-8c04-2ff60e3e0597 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:52.079964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c12c3-1831-476c-888e-46b125282b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:52.081004 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c12c3-1831-476c-888e-46b125282b4a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.082099 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c12c3-1831-476c-888e-46b125282b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.082570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c12c3-1831-476c-888e-46b125282b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.094299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c12c3-1831-476c-888e-46b125282b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:52.095987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c12c3-1831-476c-888e-46b125282b4a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-034c12c3-1831-476c-888e-46b125282b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:52.112426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:52.114263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:52.115124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:52.115124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:52.115124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:52.115307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826c963-0221-448b-ab2e-c09f7a755f8f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:52.117373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 479/968] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:52.123697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c12c3-1831-476c-888e-46b125282b4a None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:52.162753 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec516e-6e4c-4ce8-9a66-21362a44813c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:52.164181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec516e-6e4c-4ce8-9a66-21362a44813c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.164791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec516e-6e4c-4ce8-9a66-21362a44813c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.165248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec516e-6e4c-4ce8-9a66-21362a44813c None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.166275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec516e-6e4c-4ce8-9a66-21362a44813c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:52.167174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec516e-6e4c-4ce8-9a66-21362a44813c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:52.168995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec516e-6e4c-4ce8-9a66-21362a44813c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:52.180688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 490/969] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Aug 19 16:19:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:52.182245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec516e-6e4c-4ce8-9a66-21362a44813c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:52.258661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c12c3-1831-476c-888e-46b125282b4a None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:52.263218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:52.301891 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:52.308561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7bc23f-2100-49bf-95e9-946364a8040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:52.309626 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bc23f-2100-49bf-95e9-946364a8040f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.310384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bc23f-2100-49bf-95e9-946364a8040f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.310837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bc23f-2100-49bf-95e9-946364a8040f None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.311743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7bc23f-2100-49bf-95e9-946364a8040f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:52.312597 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7bc23f-2100-49bf-95e9-946364a8040f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:52.314852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7bc23f-2100-49bf-95e9-946364a8040f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:52.325250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec516e-6e4c-4ce8-9a66-21362a44813c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:52.330053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 480/970] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:52.336315 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7bc23f-2100-49bf-95e9-946364a8040f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:52.366821 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa03cf-e40a-46ff-bbb6-521f434c06ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:52.367706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa03cf-e40a-46ff-bbb6-521f434c06ca None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.368159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa03cf-e40a-46ff-bbb6-521f434c06ca None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.368581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa03cf-e40a-46ff-bbb6-521f434c06ca None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.369393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa03cf-e40a-46ff-bbb6-521f434c06ca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:52.369862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa03cf-e40a-46ff-bbb6-521f434c06ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:52.371455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa03cf-e40a-46ff-bbb6-521f434c06ca None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:52.380893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 491/971] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Aug 19 16:19:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:52.383952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa03cf-e40a-46ff-bbb6-521f434c06ca None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:52.473948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7bc23f-2100-49bf-95e9-946364a8040f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:52.494071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa03cf-e40a-46ff-bbb6-521f434c06ca None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:52.520544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:52.522535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df33a1-edd1-4bc9-8aa1-5c42fd1636f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:52.523312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df33a1-edd1-4bc9-8aa1-5c42fd1636f7 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.523664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df33a1-edd1-4bc9-8aa1-5c42fd1636f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.524002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df33a1-edd1-4bc9-8aa1-5c42fd1636f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.534383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df33a1-edd1-4bc9-8aa1-5c42fd1636f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:52.537154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df33a1-edd1-4bc9-8aa1-5c42fd1636f7 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:52.540239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0842248-f4d1-481a-a703-a4c05e964920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:52.541150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0842248-f4d1-481a-a703-a4c05e964920 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.541659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0842248-f4d1-481a-a703-a4c05e964920 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.542153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0842248-f4d1-481a-a703-a4c05e964920 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.543100 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0842248-f4d1-481a-a703-a4c05e964920 None admin] RBAC: Adding query filter params (interface=admin, service_id=a0432903fce543b6925f3b0f13805706, region_id=RegionOne) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:52.543617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0842248-f4d1-481a-a703-a4c05e964920 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:52.545319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0842248-f4d1-481a-a703-a4c05e964920 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-d3df33a1-edd1-4bc9-8aa1-5c42fd1636f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:52.550831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:52.551933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:52.552931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:52.552931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:52.552931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:52.552931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:52.552931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:52.552931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:52.552931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:52.552931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:52.552931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 16:19:52.552931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:52.558995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 481/972] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Aug 19 16:19:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:52.558995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df33a1-edd1-4bc9-8aa1-5c42fd1636f7 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:52.559589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 492/973] 10.222.0.55 () {56 vars in 1249 bytes} [Fri Aug 19 16:19:52 2022] GET /identity/v3/endpoints?service_id=a0432903fce543b6925f3b0f13805706&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:52.560707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0842248-f4d1-481a-a703-a4c05e964920 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:52.631395 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:52.680266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0842248-f4d1-481a-a703-a4c05e964920 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:52.699118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df33a1-edd1-4bc9-8aa1-5c42fd1636f7 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:52.720922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ad802-4e9a-4b53-9adf-8c3befe549ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:52.721593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ad802-4e9a-4b53-9adf-8c3befe549ee None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.721823 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ad802-4e9a-4b53-9adf-8c3befe549ee None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.722134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ad802-4e9a-4b53-9adf-8c3befe549ee None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.722741 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ad802-4e9a-4b53-9adf-8c3befe549ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:52.724115 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ad802-4e9a-4b53-9adf-8c3befe549ee None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:52.730917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 493/974] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Aug 19 16:19:52 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:19:52.741025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5864639-d3e3-409a-8d5b-585e54fc5066 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.741025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5864639-d3e3-409a-8d5b-585e54fc5066 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.741025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5864639-d3e3-409a-8d5b-585e54fc5066 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.741025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 494/975] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:52.741025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bed1e-6271-4ef8-9fc2-ade6953cbfb4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.741025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bed1e-6271-4ef8-9fc2-ade6953cbfb4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.741523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bed1e-6271-4ef8-9fc2-ade6953cbfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.746047 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea202d7e-7e67-4a2a-9c8a-500f7244e729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:52.746780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202d7e-7e67-4a2a-9c8a-500f7244e729 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.747135 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202d7e-7e67-4a2a-9c8a-500f7244e729 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.747478 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202d7e-7e67-4a2a-9c8a-500f7244e729 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.748303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea202d7e-7e67-4a2a-9c8a-500f7244e729 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:52.748683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea202d7e-7e67-4a2a-9c8a-500f7244e729 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:52.750258 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea202d7e-7e67-4a2a-9c8a-500f7244e729 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:52.761479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-079bed1e-6271-4ef8-9fc2-ade6953cbfb4 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:52.763270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-079bed1e-6271-4ef8-9fc2-ade6953cbfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:52.777159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 482/976] 10.222.0.55 () {56 vars in 1149 bytes} [Fri Aug 19 16:19:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:19:52.778686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea202d7e-7e67-4a2a-9c8a-500f7244e729 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:52.823122 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:52.856923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 495/977] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:52.878942 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ee3a8-2b31-4367-9d18-a116294d7b52 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.879283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ee3a8-2b31-4367-9d18-a116294d7b52 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.879496 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ee3a8-2b31-4367-9d18-a116294d7b52 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.881376 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 496/978] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:19:52.888377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86fdea-0bb8-4dec-885d-2ae34e5e3442 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.888929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86fdea-0bb8-4dec-885d-2ae34e5e3442 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.889373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86fdea-0bb8-4dec-885d-2ae34e5e3442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.906695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea202d7e-7e67-4a2a-9c8a-500f7244e729 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:52.908539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-fb86fdea-0bb8-4dec-885d-2ae34e5e3442 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:52.910367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fb86fdea-0bb8-4dec-885d-2ae34e5e3442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:52.919914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:52.949467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c79abf-9fd0-4254-9814-4cca0fdacd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:52.950125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c79abf-9fd0-4254-9814-4cca0fdacd02 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:52.950404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c79abf-9fd0-4254-9814-4cca0fdacd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:52.950673 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c79abf-9fd0-4254-9814-4cca0fdacd02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:52.989127 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 497/979] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:53.024816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c79abf-9fd0-4254-9814-4cca0fdacd02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:53.026540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c79abf-9fd0-4254-9814-4cca0fdacd02 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:53.131055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 483/980] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:19:52 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:19:53.136836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86fdea-0bb8-4dec-885d-2ae34e5e3442 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:53.146016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c79abf-9fd0-4254-9814-4cca0fdacd02 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:53.158402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:53.261491 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c79abf-9fd0-4254-9814-4cca0fdacd02 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:53.267267 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86fdea-0bb8-4dec-885d-2ae34e5e3442 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:53.283187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:53.299957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b877395-844c-49b9-a617-7b21bd87fd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:53.300611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b877395-844c-49b9-a617-7b21bd87fd25 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:53.301160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b877395-844c-49b9-a617-7b21bd87fd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:53.301160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b877395-844c-49b9-a617-7b21bd87fd25 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:53.301871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b877395-844c-49b9-a617-7b21bd87fd25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:53.303378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b877395-844c-49b9-a617-7b21bd87fd25 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:53.305841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: WARNING keystone.server.flask.common [None req-8b877395-844c-49b9-a617-7b21bd87fd25 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 19 16:19:53.309041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffce9a9-c943-4730-9aab-1cd84a4a2c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:53.309783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffce9a9-c943-4730-9aab-1cd84a4a2c9d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:53.310069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffce9a9-c943-4730-9aab-1cd84a4a2c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:53.310346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffce9a9-c943-4730-9aab-1cd84a4a2c9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:53.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.sql.core [None req-8b877395-844c-49b9-a617-7b21bd87fd25 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 19 16:19:53.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 19 16:19:53.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [parameters: {'user_id': '39f9a4f745864e78853e50e4b81fb1c0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 19 16:19:53.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81550) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 19 16:19:53.321883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffce9a9-c943-4730-9aab-1cd84a4a2c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:53.323381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffce9a9-c943-4730-9aab-1cd84a4a2c9d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-8b877395-844c-49b9-a617-7b21bd87fd25 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application conn.query(q) Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application result.read() Aug 19 16:19:53.336402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 16:19:53.337562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application next(self.gen) Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application yield resource Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application next(self.gen) Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self.session.rollback() Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self.force_reraise() Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise self.value Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application session.commit() Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 19 16:19:53.339140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._prepare_impl() Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self.session.flush() Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._flush(objects) Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application compat.raise_( Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application flush_context.execute() Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rec.execute(self) Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 19 16:19:53.339995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 16:19:53.340873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application conn.query(q) Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application result.read() Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 16:19:53.341694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [parameters: {'user_id': '39f9a4f745864e78853e50e4b81fb1c0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:53.343864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 19 16:19:53.346883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 19 16:19:53.349126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 19 16:19:53.349126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 19 16:19:53.349126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 19 16:19:53.349126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 19 16:19:53.349126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 19 16:19:53.349126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:53.349126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 484/981] 10.222.0.55 () {60 vars in 1131 bytes} [Fri Aug 19 16:19:53 2022] POST /identity/v3/users => generated 160 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:53.349126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b877395-844c-49b9-a617-7b21bd87fd25 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4ffce9a9-c943-4730-9aab-1cd84a4a2c9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:19:53.350154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:53.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:19:53.353174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:19:53.353174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:19:53.353174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:53.353174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 498/982] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:53.353174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffce9a9-c943-4730-9aab-1cd84a4a2c9d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:53.419967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:53.451936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b877395-844c-49b9-a617-7b21bd87fd25 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:53.459254 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffce9a9-c943-4730-9aab-1cd84a4a2c9d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:53.489299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bd2a4-76a2-496b-98e7-d141c97ca1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:53.489915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bd2a4-76a2-496b-98e7-d141c97ca1e1 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:53.490202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bd2a4-76a2-496b-98e7-d141c97ca1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:53.490476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bd2a4-76a2-496b-98e7-d141c97ca1e1 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:53.491122 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bd2a4-76a2-496b-98e7-d141c97ca1e1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:53.492742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bd2a4-76a2-496b-98e7-d141c97ca1e1 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:53.493967 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389a9b0-5509-42cb-b5b9-1e5b602bdf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:53.494598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a9b0-5509-42cb-b5b9-1e5b602bdf17 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:53.495125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a9b0-5509-42cb-b5b9-1e5b602bdf17 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:53.495125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a9b0-5509-42cb-b5b9-1e5b602bdf17 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:53.503843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389a9b0-5509-42cb-b5b9-1e5b602bdf17 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:53.505308 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389a9b0-5509-42cb-b5b9-1e5b602bdf17 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:53.516956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 485/983] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:19:53 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:53.516956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bd2a4-76a2-496b-98e7-d141c97ca1e1 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a389a9b0-5509-42cb-b5b9-1e5b602bdf17 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:19:53.517210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:53.518213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:53.519914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:53.519914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:53.519914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:19:53.519914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:19:53.519914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:19:53.519914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:19:53.519914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:19:53.519914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:19:53.519914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 16:19:53.519914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:53.523234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 499/984] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:19:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:53.524211 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389a9b0-5509-42cb-b5b9-1e5b602bdf17 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:53.554742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:53.643682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389a9b0-5509-42cb-b5b9-1e5b602bdf17 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:53.655023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bd2a4-76a2-496b-98e7-d141c97ca1e1 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:53.686706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728260a-1973-43c1-b302-f2ef6d7a2662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:53.687249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728260a-1973-43c1-b302-f2ef6d7a2662 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:53.687525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728260a-1973-43c1-b302-f2ef6d7a2662 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:53.687797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728260a-1973-43c1-b302-f2ef6d7a2662 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:53.688906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728260a-1973-43c1-b302-f2ef6d7a2662 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:53.689212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728260a-1973-43c1-b302-f2ef6d7a2662 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:53.689896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:53.691792 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728260a-1973-43c1-b302-f2ef6d7a2662 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:53.699847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dd764-928f-4290-bbbe-93774061d383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:53.700459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dd764-928f-4290-bbbe-93774061d383 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:53.700735 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dd764-928f-4290-bbbe-93774061d383 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:53.701031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dd764-928f-4290-bbbe-93774061d383 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:53.702128 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dd764-928f-4290-bbbe-93774061d383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:53.702128 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dd764-928f-4290-bbbe-93774061d383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:53.703348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dd764-928f-4290-bbbe-93774061d383 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:53.716475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 486/985] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:53.723280 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dd764-928f-4290-bbbe-93774061d383 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:53.734867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 500/986] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Aug 19 16:19:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:53.837174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dd764-928f-4290-bbbe-93774061d383 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:53.875432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d15e51-31f9-4b0b-bba7-58660e7528d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:53.876074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d15e51-31f9-4b0b-bba7-58660e7528d4 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:53.876348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d15e51-31f9-4b0b-bba7-58660e7528d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:53.876621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d15e51-31f9-4b0b-bba7-58660e7528d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:53.884583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d15e51-31f9-4b0b-bba7-58660e7528d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:53.886055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d15e51-31f9-4b0b-bba7-58660e7528d4 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-96d15e51-31f9-4b0b-bba7-58660e7528d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:19:53.897849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:53.898736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:53.900046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:19:53.900046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:19:53.900046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:19:53.900046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:19:53.900046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:19:53.900046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:19:53.900046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:19:53.900046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:53.900046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 487/987] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:53.904988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728260a-1973-43c1-b302-f2ef6d7a2662 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:54.001206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:54.024941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728260a-1973-43c1-b302-f2ef6d7a2662 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:54.061145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51505a36-b657-46f7-8e21-79cf59624ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:54.061915 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51505a36-b657-46f7-8e21-79cf59624ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:54.062296 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51505a36-b657-46f7-8e21-79cf59624ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:54.062669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51505a36-b657-46f7-8e21-79cf59624ca9 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:54.063457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51505a36-b657-46f7-8e21-79cf59624ca9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:54.063863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51505a36-b657-46f7-8e21-79cf59624ca9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:54.065506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51505a36-b657-46f7-8e21-79cf59624ca9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:54.078493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 501/988] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:19:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:54.085244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d15e51-31f9-4b0b-bba7-58660e7528d4 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:54.134252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:54.185374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d15e51-31f9-4b0b-bba7-58660e7528d4 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:54.219787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5416c7-b788-4ede-8c17-9bc88901148b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:54.220538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5416c7-b788-4ede-8c17-9bc88901148b None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:54.220949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5416c7-b788-4ede-8c17-9bc88901148b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:54.221320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5416c7-b788-4ede-8c17-9bc88901148b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:54.222319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5416c7-b788-4ede-8c17-9bc88901148b None admin] RBAC: Adding query filter params (role.id=2dc5eb239bdc4aa6be297199d70bb634, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=88694a50b9f04487a521ba6df110d976) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:54.222728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5416c7-b788-4ede-8c17-9bc88901148b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:54.224322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5416c7-b788-4ede-8c17-9bc88901148b None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:54.230889 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 488/989] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:19:54 2022] GET /identity/v3/role_assignments?user.id=88694a50b9f04487a521ba6df110d976&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=2dc5eb239bdc4aa6be297199d70bb634 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:54.781093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bac5a4-6228-4e3f-b5bd-8a57bebb3a36 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:54.781628 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bac5a4-6228-4e3f-b5bd-8a57bebb3a36 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:54.781628 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bac5a4-6228-4e3f-b5bd-8a57bebb3a36 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:54.783158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 502/990] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:54.788267 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283cc92-28bf-4684-bade-e579cdf664c5 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:54.788663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283cc92-28bf-4684-bade-e579cdf664c5 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:54.788891 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283cc92-28bf-4684-bade-e579cdf664c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:54.807862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b283cc92-28bf-4684-bade-e579cdf664c5 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:54.809353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b283cc92-28bf-4684-bade-e579cdf664c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:54.876577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 489/991] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:55.145755 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bac5a4-6228-4e3f-b5bd-8a57bebb3a36 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:55.251289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bac5a4-6228-4e3f-b5bd-8a57bebb3a36 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:55.285984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2d585-6218-44ac-823e-14e64beceb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:55.286611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2d585-6218-44ac-823e-14e64beceb88 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:55.286888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2d585-6218-44ac-823e-14e64beceb88 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:55.287157 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2d585-6218-44ac-823e-14e64beceb88 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:55.287732 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2d585-6218-44ac-823e-14e64beceb88 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:55.289113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2d585-6218-44ac-823e-14e64beceb88 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c4e2d585-6218-44ac-823e-14e64beceb88 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:55.297744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:55.298714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:55.299565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 503/992] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Aug 19 16:19:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:55.306958 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283cc92-28bf-4684-bade-e579cdf664c5 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:55.352416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:55.410477 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283cc92-28bf-4684-bade-e579cdf664c5 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:55.443441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a88d68-4483-4517-8f8e-11237d027649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:55.444047 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a88d68-4483-4517-8f8e-11237d027649 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:55.444323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a88d68-4483-4517-8f8e-11237d027649 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:55.444593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a88d68-4483-4517-8f8e-11237d027649 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:55.445254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a88d68-4483-4517-8f8e-11237d027649 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:55.445553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a88d68-4483-4517-8f8e-11237d027649 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:55.446904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a88d68-4483-4517-8f8e-11237d027649 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:55.451863 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 490/993] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Aug 19 16:19:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:55.458479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2d585-6218-44ac-823e-14e64beceb88 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:55.468097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:55.483568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df01a02-0258-44e8-9fbb-82ec1eefc071 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:55.483861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df01a02-0258-44e8-9fbb-82ec1eefc071 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:55.484143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df01a02-0258-44e8-9fbb-82ec1eefc071 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:55.485915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 491/994] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:19:55.490794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715938b1-a5dd-4946-9cdd-6f453a9d486d None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:55.491077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715938b1-a5dd-4946-9cdd-6f453a9d486d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:55.491349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715938b1-a5dd-4946-9cdd-6f453a9d486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:55.510183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-715938b1-a5dd-4946-9cdd-6f453a9d486d None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:55.511351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-715938b1-a5dd-4946-9cdd-6f453a9d486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:55.582331 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 492/995] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:55.585630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2d585-6218-44ac-823e-14e64beceb88 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:55.621125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ecc6b-3093-4b57-8acd-3ea6ad308828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:55.621742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ecc6b-3093-4b57-8acd-3ea6ad308828 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:55.622038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ecc6b-3093-4b57-8acd-3ea6ad308828 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:55.622346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ecc6b-3093-4b57-8acd-3ea6ad308828 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:55.622978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ecc6b-3093-4b57-8acd-3ea6ad308828 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:55.623280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ecc6b-3093-4b57-8acd-3ea6ad308828 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:55.624655 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ecc6b-3093-4b57-8acd-3ea6ad308828 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:55.635018 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 504/996] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Aug 19 16:19:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:55.636643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715938b1-a5dd-4946-9cdd-6f453a9d486d None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:55.676661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:55.744498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715938b1-a5dd-4946-9cdd-6f453a9d486d None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:55.779230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc6890-a8fa-4964-92e5-a4bd0969e675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:55.779835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc6890-a8fa-4964-92e5-a4bd0969e675 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:55.780111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc6890-a8fa-4964-92e5-a4bd0969e675 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:55.780378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc6890-a8fa-4964-92e5-a4bd0969e675 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:55.781079 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc6890-a8fa-4964-92e5-a4bd0969e675 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:55.782430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc6890-a8fa-4964-92e5-a4bd0969e675 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:55.798736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 493/997] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:19:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:19:55.846670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ecc6b-3093-4b57-8acd-3ea6ad308828 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:55.925785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:55.959948 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ecc6b-3093-4b57-8acd-3ea6ad308828 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:56.001225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359e85f-badb-4c3a-a739-8682e11fc1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:56.001858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359e85f-badb-4c3a-a739-8682e11fc1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:56.002132 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359e85f-badb-4c3a-a739-8682e11fc1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:56.002398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359e85f-badb-4c3a-a739-8682e11fc1e2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:56.002969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359e85f-badb-4c3a-a739-8682e11fc1e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:56.004884 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359e85f-badb-4c3a-a739-8682e11fc1e2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-f359e85f-badb-4c3a-a739-8682e11fc1e2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:56.013621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:56.014565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:56.015774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 505/998] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:56.020809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc6890-a8fa-4964-92e5-a4bd0969e675 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:56.115469 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:56.122053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc6890-a8fa-4964-92e5-a4bd0969e675 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:56.155378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f773538-492d-4f94-b7b6-4305908ae5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:56.155986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f773538-492d-4f94-b7b6-4305908ae5e5 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:56.156260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f773538-492d-4f94-b7b6-4305908ae5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:56.156525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f773538-492d-4f94-b7b6-4305908ae5e5 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:56.157185 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f773538-492d-4f94-b7b6-4305908ae5e5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:56.157483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f773538-492d-4f94-b7b6-4305908ae5e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:56.158835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f773538-492d-4f94-b7b6-4305908ae5e5 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:56.163989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 494/999] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:56.169463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359e85f-badb-4c3a-a739-8682e11fc1e2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:56.185648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87482fc9-236f-4932-a874-36214e365030 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:56.185947 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87482fc9-236f-4932-a874-36214e365030 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:56.186218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87482fc9-236f-4932-a874-36214e365030 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:56.187684 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 495/1000] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:56.193229 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d60f78-8116-4da2-ab3c-370916ef205c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:56.193527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d60f78-8116-4da2-ab3c-370916ef205c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:56.193808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d60f78-8116-4da2-ab3c-370916ef205c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:56.215521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-46d60f78-8116-4da2-ab3c-370916ef205c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:56.217000 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-46d60f78-8116-4da2-ab3c-370916ef205c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:56.276047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359e85f-badb-4c3a-a739-8682e11fc1e2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:56.290827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 496/1001] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:56 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:56.308093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:56.314642 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb767b-9fe6-449d-aa76-e1a3a66b20cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:56.315252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb767b-9fe6-449d-aa76-e1a3a66b20cb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:56.315530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb767b-9fe6-449d-aa76-e1a3a66b20cb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:56.316351 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb767b-9fe6-449d-aa76-e1a3a66b20cb None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:56.317002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb767b-9fe6-449d-aa76-e1a3a66b20cb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:56.317304 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb767b-9fe6-449d-aa76-e1a3a66b20cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:56.318669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb767b-9fe6-449d-aa76-e1a3a66b20cb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:56.328355 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 506/1002] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:56.329938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d60f78-8116-4da2-ab3c-370916ef205c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:56.420458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:56.431172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d60f78-8116-4da2-ab3c-370916ef205c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:56.464865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f8544-e870-461a-9265-eeda11731711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:56.465470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f8544-e870-461a-9265-eeda11731711 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:56.465799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f8544-e870-461a-9265-eeda11731711 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:56.466040 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f8544-e870-461a-9265-eeda11731711 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:56.466698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f8544-e870-461a-9265-eeda11731711 None admin] RBAC: Adding query filter params (interface=public, service_id=5936c0b06b364b71a765549fcce5cb5d, region_id=RegionOne) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:56.466998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f8544-e870-461a-9265-eeda11731711 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:56.468343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f8544-e870-461a-9265-eeda11731711 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:56.474145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 497/1003] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Aug 19 16:19:56 2022] GET /identity/v3/endpoints?service_id=5936c0b06b364b71a765549fcce5cb5d&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:56.479674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb767b-9fe6-449d-aa76-e1a3a66b20cb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:56.558708 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f8544-e870-461a-9265-eeda11731711 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:56.584677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb767b-9fe6-449d-aa76-e1a3a66b20cb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:56.627114 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711e32d-1dbe-46f6-a1a8-26cc8da97237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:56.627881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711e32d-1dbe-46f6-a1a8-26cc8da97237 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:56.628209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711e32d-1dbe-46f6-a1a8-26cc8da97237 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:56.628487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711e32d-1dbe-46f6-a1a8-26cc8da97237 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:56.629120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711e32d-1dbe-46f6-a1a8-26cc8da97237 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:56.630495 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711e32d-1dbe-46f6-a1a8-26cc8da97237 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:56.636781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 507/1004] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Aug 19 16:19:56 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:19:56.669439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f8544-e870-461a-9265-eeda11731711 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:56.711327 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425180c-71eb-4e3f-a211-ca2527d20489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:56.711977 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425180c-71eb-4e3f-a211-ca2527d20489 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:56.712254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425180c-71eb-4e3f-a211-ca2527d20489 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:56.712523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425180c-71eb-4e3f-a211-ca2527d20489 None admin] PATH_INFO: `/v3/services/image` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:56.713144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425180c-71eb-4e3f-a211-ca2527d20489 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:56.714543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425180c-71eb-4e3f-a211-ca2527d20489 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-3425180c-71eb-4e3f-a211-ca2527d20489 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:56.725102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:56.726110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:19:56.727624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 498/1005] 10.222.0.55 () {56 vars in 1105 bytes} [Fri Aug 19 16:19:56 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:19:56.732302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711e32d-1dbe-46f6-a1a8-26cc8da97237 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:56.800385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:56.852316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711e32d-1dbe-46f6-a1a8-26cc8da97237 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:56.885873 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdccde-aa08-4bfb-ae8f-9345072b32e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:56.886482 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdccde-aa08-4bfb-ae8f-9345072b32e4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:56.886759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdccde-aa08-4bfb-ae8f-9345072b32e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:56.887028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdccde-aa08-4bfb-ae8f-9345072b32e4 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:56.887646 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdccde-aa08-4bfb-ae8f-9345072b32e4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:56.887942 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdccde-aa08-4bfb-ae8f-9345072b32e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:56.889407 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdccde-aa08-4bfb-ae8f-9345072b32e4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:56.895060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 508/1006] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:56.900516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425180c-71eb-4e3f-a211-ca2527d20489 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:56.922788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:57.009594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425180c-71eb-4e3f-a211-ca2527d20489 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:57.044337 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d73830-d21a-470e-96b2-c75c3a8ddb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:57.044968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d73830-d21a-470e-96b2-c75c3a8ddb7e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:57.045248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d73830-d21a-470e-96b2-c75c3a8ddb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:57.045518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d73830-d21a-470e-96b2-c75c3a8ddb7e None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:57.046141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d73830-d21a-470e-96b2-c75c3a8ddb7e None admin] RBAC: Adding query filter params (type=image) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:57.046439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d73830-d21a-470e-96b2-c75c3a8ddb7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:57.047796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d73830-d21a-470e-96b2-c75c3a8ddb7e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:57.060792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 499/1007] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:57.783309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6f08a-bd78-48f1-a39d-8250fd73c72c None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:57.783682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6f08a-bd78-48f1-a39d-8250fd73c72c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:57.783887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6f08a-bd78-48f1-a39d-8250fd73c72c None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:57.785908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 509/1008] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:57.790871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b76de-6b6f-4161-ab98-0e2e09825f85 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:57.791161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b76de-6b6f-4161-ab98-0e2e09825f85 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:57.791549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b76de-6b6f-4161-ab98-0e2e09825f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:57.809943 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d11b76de-6b6f-4161-ab98-0e2e09825f85 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:57.811119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d11b76de-6b6f-4161-ab98-0e2e09825f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:57.877837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 500/1009] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:57 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:58.144440 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6f08a-bd78-48f1-a39d-8250fd73c72c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:58.247225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6f08a-bd78-48f1-a39d-8250fd73c72c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:58.270062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:58.282371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e18d16-9f1e-44f5-9c5a-ca256f4a3e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:58.282987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e18d16-9f1e-44f5-9c5a-ca256f4a3e69 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:58.283265 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e18d16-9f1e-44f5-9c5a-ca256f4a3e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:58.283536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e18d16-9f1e-44f5-9c5a-ca256f4a3e69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:58.285363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e18d16-9f1e-44f5-9c5a-ca256f4a3e69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:58.286823 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e18d16-9f1e-44f5-9c5a-ca256f4a3e69 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:58.288588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 510/1010] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:19:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:58.294252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b76de-6b6f-4161-ab98-0e2e09825f85 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:58.308549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:58.397581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b76de-6b6f-4161-ab98-0e2e09825f85 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:58.431995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecce99d-87ac-4a85-944a-bded980e924a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:58.432607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecce99d-87ac-4a85-944a-bded980e924a None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:58.432914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecce99d-87ac-4a85-944a-bded980e924a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:58.433183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecce99d-87ac-4a85-944a-bded980e924a None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:58.433828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecce99d-87ac-4a85-944a-bded980e924a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:58.435246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecce99d-87ac-4a85-944a-bded980e924a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:58.468341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 501/1011] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:19:58 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:19:58.562740 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc542dd2-e274-4e77-831b-055103979dfe None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:58.563025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc542dd2-e274-4e77-831b-055103979dfe None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:58.563302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc542dd2-e274-4e77-831b-055103979dfe None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:58.564660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 511/1012] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:58.570889 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e02453-955d-4518-adbd-2711dd2e9302 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:58.571172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e02453-955d-4518-adbd-2711dd2e9302 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:58.571449 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e02453-955d-4518-adbd-2711dd2e9302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:58.591017 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-55e02453-955d-4518-adbd-2711dd2e9302 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:58.592224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-55e02453-955d-4518-adbd-2711dd2e9302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:58.656412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:58.668897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 502/1013] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:19:58.933615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc542dd2-e274-4e77-831b-055103979dfe None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:58.983032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c3ae7-1f46-45be-9998-403a376d7ab3 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:58.983356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c3ae7-1f46-45be-9998-403a376d7ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:58.983593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c3ae7-1f46-45be-9998-403a376d7ab3 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:58.984974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 503/1014] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:19:58.992010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507399c3-db99-407f-b9f1-4e8832fc1633 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:58.992368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507399c3-db99-407f-b9f1-4e8832fc1633 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:58.992718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507399c3-db99-407f-b9f1-4e8832fc1633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:59.011550 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-507399c3-db99-407f-b9f1-4e8832fc1633 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:19:59.013006 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-507399c3-db99-407f-b9f1-4e8832fc1633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:19:59.047837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc542dd2-e274-4e77-831b-055103979dfe None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:59.085460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb6c3f-2d68-4d98-a83f-df6606928dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:59.086318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb6c3f-2d68-4d98-a83f-df6606928dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:59.086428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 504/1015] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:19:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:19:59.086528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb6c3f-2d68-4d98-a83f-df6606928dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:59.086617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb6c3f-2d68-4d98-a83f-df6606928dd4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:59.087218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb6c3f-2d68-4d98-a83f-df6606928dd4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:59.088588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb6c3f-2d68-4d98-a83f-df6606928dd4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-d2bb6c3f-2d68-4d98-a83f-df6606928dd4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:19:59.097928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:19:59.098904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:19:59.099744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 512/1016] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:19:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:19:59.104955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507399c3-db99-407f-b9f1-4e8832fc1633 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:59.210928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507399c3-db99-407f-b9f1-4e8832fc1633 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:59.235466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:59.247920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfcd2d-f515-490a-82d4-cd7f169cc663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:59.248818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfcd2d-f515-490a-82d4-cd7f169cc663 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:59.249328 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfcd2d-f515-490a-82d4-cd7f169cc663 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:59.249835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfcd2d-f515-490a-82d4-cd7f169cc663 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:59.250658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfcd2d-f515-490a-82d4-cd7f169cc663 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:59.251178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfcd2d-f515-490a-82d4-cd7f169cc663 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:59.252762 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfcd2d-f515-490a-82d4-cd7f169cc663 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:59.259351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 505/1017] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:59.264856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb6c3f-2d68-4d98-a83f-df6606928dd4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:59.325048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:59.358055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfcd2d-f515-490a-82d4-cd7f169cc663 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:59.373232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb6c3f-2d68-4d98-a83f-df6606928dd4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:59.415155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa2f46-30fd-4dfb-abe8-5c08e2e17bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:59.415771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa2f46-30fd-4dfb-abe8-5c08e2e17bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:59.416129 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa2f46-30fd-4dfb-abe8-5c08e2e17bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:59.416339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa2f46-30fd-4dfb-abe8-5c08e2e17bf4 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:59.417117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa2f46-30fd-4dfb-abe8-5c08e2e17bf4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:19:59.417585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa2f46-30fd-4dfb-abe8-5c08e2e17bf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:59.418795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa2f46-30fd-4dfb-abe8-5c08e2e17bf4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:59.429236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 513/1018] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Aug 19 16:19:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:19:59.430241 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa2f46-30fd-4dfb-abe8-5c08e2e17bf4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:19:59.466202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfcd2d-f515-490a-82d4-cd7f169cc663 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:59.507310 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acf56d-9364-4188-a166-c5601996d1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:59.507926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acf56d-9364-4188-a166-c5601996d1de None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:59.508206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acf56d-9364-4188-a166-c5601996d1de None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:59.508482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acf56d-9364-4188-a166-c5601996d1de None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:59.509585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acf56d-9364-4188-a166-c5601996d1de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:59.511336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acf56d-9364-4188-a166-c5601996d1de None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:59.534717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa2f46-30fd-4dfb-abe8-5c08e2e17bf4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:19:59.537006 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 506/1019] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:19:59 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:19:59.562943 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:19:59.594793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1c86d-a9a0-4563-b75b-60ae0f1000f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:19:59.595816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1c86d-a9a0-4563-b75b-60ae0f1000f3 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:19:59.596292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1c86d-a9a0-4563-b75b-60ae0f1000f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:19:59.597346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1c86d-a9a0-4563-b75b-60ae0f1000f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:19:59.598552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1c86d-a9a0-4563-b75b-60ae0f1000f3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:19:59.601087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1c86d-a9a0-4563-b75b-60ae0f1000f3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:19:59.640792 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 514/1020] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:19:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:20:00.473808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ac592-7a71-4104-bb7d-13623ca22364 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:00.474327 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ac592-7a71-4104-bb7d-13623ca22364 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:00.474327 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ac592-7a71-4104-bb7d-13623ca22364 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:00.475706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 507/1021] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:00.480987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4e8c8-ff90-4fc5-be36-7df818510f41 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:00.481362 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4e8c8-ff90-4fc5-be36-7df818510f41 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:00.481528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4e8c8-ff90-4fc5-be36-7df818510f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:00.499988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a1e4e8c8-ff90-4fc5-be36-7df818510f41 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:00.501201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a1e4e8c8-ff90-4fc5-be36-7df818510f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:00.579553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 515/1022] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:00.860197 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ac592-7a71-4104-bb7d-13623ca22364 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:00.912038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:00.965880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ac592-7a71-4104-bb7d-13623ca22364 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:01.000777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0742a-cfdb-4fa0-923a-a65aa4f79abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:01.001436 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0742a-cfdb-4fa0-923a-a65aa4f79abe None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:01.001713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0742a-cfdb-4fa0-923a-a65aa4f79abe None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:01.001980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0742a-cfdb-4fa0-923a-a65aa4f79abe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:01.003116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0742a-cfdb-4fa0-923a-a65aa4f79abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:01.004527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0742a-cfdb-4fa0-923a-a65aa4f79abe None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:01.006320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 508/1023] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:01.011756 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4e8c8-ff90-4fc5-be36-7df818510f41 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:01.106775 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:01.115360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4e8c8-ff90-4fc5-be36-7df818510f41 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:01.149696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37a54d-a383-48a2-82aa-965229322697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:01.150308 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37a54d-a383-48a2-82aa-965229322697 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:01.150585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37a54d-a383-48a2-82aa-965229322697 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:01.150854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37a54d-a383-48a2-82aa-965229322697 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:01.151505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37a54d-a383-48a2-82aa-965229322697 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:01.152973 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37a54d-a383-48a2-82aa-965229322697 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:01.177511 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 516/1024] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:20:01 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:20:01.478639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaeeb6-8f0b-4c29-bac5-78fd0a7fe685 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:01.479145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaeeb6-8f0b-4c29-bac5-78fd0a7fe685 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:01.479145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaeeb6-8f0b-4c29-bac5-78fd0a7fe685 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:01.480567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 509/1025] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:01.485525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca61f46-7949-426d-b44d-b64d9ae4ead9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:01.485901 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca61f46-7949-426d-b44d-b64d9ae4ead9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:01.486085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca61f46-7949-426d-b44d-b64d9ae4ead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:01.504182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2ca61f46-7949-426d-b44d-b64d9ae4ead9 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:01.505413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2ca61f46-7949-426d-b44d-b64d9ae4ead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:01.575767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 517/1026] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:01.638285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a0b37-b46c-4ab7-b56b-3446efd21833 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:01.638596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a0b37-b46c-4ab7-b56b-3446efd21833 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:01.638829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a0b37-b46c-4ab7-b56b-3446efd21833 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:01.640830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 510/1027] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:01.646400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657de233-e721-447f-85dd-742a6d115bfa None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:01.646688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657de233-e721-447f-85dd-742a6d115bfa None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:01.646961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657de233-e721-447f-85dd-742a6d115bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:01.667841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-657de233-e721-447f-85dd-742a6d115bfa None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:01.669074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-657de233-e721-447f-85dd-742a6d115bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:01.738816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 518/1028] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:01.841181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a0b37-b46c-4ab7-b56b-3446efd21833 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:01.943127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a0b37-b46c-4ab7-b56b-3446efd21833 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:01.959014 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:01.977570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff4e11-e1ad-4514-9e0c-a8f9f872d030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:01.978194 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff4e11-e1ad-4514-9e0c-a8f9f872d030 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:01.978511 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff4e11-e1ad-4514-9e0c-a8f9f872d030 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:01.978734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff4e11-e1ad-4514-9e0c-a8f9f872d030 None admin] PATH_INFO: `/v3/services/image` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:01.979303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff4e11-e1ad-4514-9e0c-a8f9f872d030 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:01.980654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff4e11-e1ad-4514-9e0c-a8f9f872d030 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-d8ff4e11-e1ad-4514-9e0c-a8f9f872d030 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:01.990076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:01.991030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 16:20:01.991868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:01.995994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 511/1029] 10.222.0.55 () {56 vars in 1105 bytes} [Fri Aug 19 16:20:01 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:01.997076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657de233-e721-447f-85dd-742a6d115bfa None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:02.014459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:02.014923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff4e11-e1ad-4514-9e0c-a8f9f872d030 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:02.108001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657de233-e721-447f-85dd-742a6d115bfa None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:02.123078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff4e11-e1ad-4514-9e0c-a8f9f872d030 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:02.142732 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600d485-ea39-4639-b207-076184ee0c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:02.143344 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600d485-ea39-4639-b207-076184ee0c5a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:02.144144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600d485-ea39-4639-b207-076184ee0c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:02.144569 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600d485-ea39-4639-b207-076184ee0c5a None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:02.145229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600d485-ea39-4639-b207-076184ee0c5a None admin] RBAC: Adding query filter params (name=image) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:02.145533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600d485-ea39-4639-b207-076184ee0c5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:02.146904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600d485-ea39-4639-b207-076184ee0c5a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:02.152825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 519/1030] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:02.158214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600d485-ea39-4639-b207-076184ee0c5a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:02.158963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fa898-331a-4c00-bdc8-b424cd5490a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:02.160145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fa898-331a-4c00-bdc8-b424cd5490a5 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:02.160420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fa898-331a-4c00-bdc8-b424cd5490a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:02.160690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fa898-331a-4c00-bdc8-b424cd5490a5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:02.173193 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fa898-331a-4c00-bdc8-b424cd5490a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-274fa898-331a-4c00-bdc8-b424cd5490a5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:20:02.178319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:02.179240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:02.180072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:02.180072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:02.180072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:02.180072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:02.180072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:20:02.180072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:20:02.180072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:20:02.180072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:20:02.180072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:20:02.180072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:20:02.180072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 16:20:02.180072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:02.184667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 512/1031] 10.222.0.55 () {56 vars in 1120 bytes} [Fri Aug 19 16:20:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:20:02.185523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fa898-331a-4c00-bdc8-b424cd5490a5 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:02.230818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:02.244297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:02.269677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600d485-ea39-4639-b207-076184ee0c5a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:02.294733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fa898-331a-4c00-bdc8-b424cd5490a5 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:02.306657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab54c10-b1ee-4b88-9cde-aadff74b0a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:02.307290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab54c10-b1ee-4b88-9cde-aadff74b0a44 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:02.307569 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab54c10-b1ee-4b88-9cde-aadff74b0a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:02.307843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab54c10-b1ee-4b88-9cde-aadff74b0a44 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:02.308493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab54c10-b1ee-4b88-9cde-aadff74b0a44 None admin] RBAC: Adding query filter params (type=image) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:02.308825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab54c10-b1ee-4b88-9cde-aadff74b0a44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:02.310254 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab54c10-b1ee-4b88-9cde-aadff74b0a44 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:02.316633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 520/1032] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:02.323438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab54c10-b1ee-4b88-9cde-aadff74b0a44 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:02.337864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac28de-024f-40b0-99c5-dcd2dab9b76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:02.338472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac28de-024f-40b0-99c5-dcd2dab9b76b None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:02.338748 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac28de-024f-40b0-99c5-dcd2dab9b76b None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:02.339017 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac28de-024f-40b0-99c5-dcd2dab9b76b None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:02.339666 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac28de-024f-40b0-99c5-dcd2dab9b76b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:02.340206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac28de-024f-40b0-99c5-dcd2dab9b76b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:02.341389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac28de-024f-40b0-99c5-dcd2dab9b76b None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:02.349678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 513/1033] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Aug 19 16:20:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:02.355723 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac28de-024f-40b0-99c5-dcd2dab9b76b None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:02.440171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab54c10-b1ee-4b88-9cde-aadff74b0a44 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:02.460035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac28de-024f-40b0-99c5-dcd2dab9b76b None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:02.461734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:02.478685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9969b5-1111-4c89-8547-36e8fbb4d212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:02.479322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9969b5-1111-4c89-8547-36e8fbb4d212 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:02.479619 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9969b5-1111-4c89-8547-36e8fbb4d212 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:02.479918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9969b5-1111-4c89-8547-36e8fbb4d212 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:02.480635 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9969b5-1111-4c89-8547-36e8fbb4d212 None admin] RBAC: Adding query filter params (interface=public, service_id=8b011f1d3506410284347ce52c61b667, region_id=RegionOne) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:02.481015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9969b5-1111-4c89-8547-36e8fbb4d212 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:02.482352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9969b5-1111-4c89-8547-36e8fbb4d212 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:02.492383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 521/1034] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Aug 19 16:20:02 2022] GET /identity/v3/endpoints?service_id=8b011f1d3506410284347ce52c61b667&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:02.493303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9969b5-1111-4c89-8547-36e8fbb4d212 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:02.498151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7540853-93b7-4352-bf13-1cb6bd360d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:02.498557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:02.499426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7540853-93b7-4352-bf13-1cb6bd360d4f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:02.499929 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7540853-93b7-4352-bf13-1cb6bd360d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:02.500225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7540853-93b7-4352-bf13-1cb6bd360d4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:02.512747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7540853-93b7-4352-bf13-1cb6bd360d4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:02.514217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7540853-93b7-4352-bf13-1cb6bd360d4f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-e7540853-93b7-4352-bf13-1cb6bd360d4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:20:02.529285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:02.530227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:20:02.531446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:20:02.531446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:02.531446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:02.531446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 514/1035] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:02.536404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7540853-93b7-4352-bf13-1cb6bd360d4f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:02.602879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9969b5-1111-4c89-8547-36e8fbb4d212 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:02.637403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab85383-ce66-4db1-bd7d-e06888372b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:02.638015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab85383-ce66-4db1-bd7d-e06888372b78 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:02.638299 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab85383-ce66-4db1-bd7d-e06888372b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:02.638578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab85383-ce66-4db1-bd7d-e06888372b78 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:02.639190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab85383-ce66-4db1-bd7d-e06888372b78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:02.640572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab85383-ce66-4db1-bd7d-e06888372b78 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:02.645295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7540853-93b7-4352-bf13-1cb6bd360d4f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:02.650177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 522/1036] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Aug 19 16:20:02 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:20:02.686317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b8460-c83c-495a-a985-c0fd01d4e9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:02.686948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b8460-c83c-495a-a985-c0fd01d4e9ea None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:02.687227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b8460-c83c-495a-a985-c0fd01d4e9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:02.687497 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b8460-c83c-495a-a985-c0fd01d4e9ea None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:02.688142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b8460-c83c-495a-a985-c0fd01d4e9ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:02.688442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b8460-c83c-495a-a985-c0fd01d4e9ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:02.689867 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b8460-c83c-495a-a985-c0fd01d4e9ea None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:02.708841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 515/1037] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:02.710141 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab85383-ce66-4db1-bd7d-e06888372b78 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:02.783810 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:02.821257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab85383-ce66-4db1-bd7d-e06888372b78 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:02.855905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e028a2c-108d-4b9f-93b2-f0780df3daca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:02.856546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e028a2c-108d-4b9f-93b2-f0780df3daca None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:02.856854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e028a2c-108d-4b9f-93b2-f0780df3daca None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:02.857131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e028a2c-108d-4b9f-93b2-f0780df3daca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:02.865733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e028a2c-108d-4b9f-93b2-f0780df3daca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:02.867145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e028a2c-108d-4b9f-93b2-f0780df3daca None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-7e028a2c-108d-4b9f-93b2-f0780df3daca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:20:02.879301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:02.880276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:02.881280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:02.881280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:02.881280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:20:02.881280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:20:02.881280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:20:02.881280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:20:02.881280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:20:02.881280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:20:02.881280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:20:02.881280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:02.881280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 523/1038] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Aug 19 16:20:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:02.887628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b8460-c83c-495a-a985-c0fd01d4e9ea None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:02.903704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:02.990402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b8460-c83c-495a-a985-c0fd01d4e9ea None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:03.025425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3b8ab-fadb-4161-9298-ccfa66ca78d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:03.026038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3b8ab-fadb-4161-9298-ccfa66ca78d6 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:03.026332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3b8ab-fadb-4161-9298-ccfa66ca78d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:03.026589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3b8ab-fadb-4161-9298-ccfa66ca78d6 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:03.027315 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3b8ab-fadb-4161-9298-ccfa66ca78d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:03.027619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3b8ab-fadb-4161-9298-ccfa66ca78d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:03.029085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3b8ab-fadb-4161-9298-ccfa66ca78d6 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:03.051418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 516/1039] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Aug 19 16:20:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:03.052685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e028a2c-108d-4b9f-93b2-f0780df3daca None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:03.155159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e028a2c-108d-4b9f-93b2-f0780df3daca None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:03.155595 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4caa2dc-1bda-47d5-8756-4822bcc9639d None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:03.155883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4caa2dc-1bda-47d5-8756-4822bcc9639d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:03.156156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4caa2dc-1bda-47d5-8756-4822bcc9639d None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:03.157773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 517/1040] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:20:03.163575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfaae4-c5d1-4376-b369-f0c0b31023fc None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:03.163864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfaae4-c5d1-4376-b369-f0c0b31023fc None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:03.164144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfaae4-c5d1-4376-b369-f0c0b31023fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:03.182457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d9dfaae4-c5d1-4376-b369-f0c0b31023fc None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:03.183654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d9dfaae4-c5d1-4376-b369-f0c0b31023fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:03.184497 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:03.193192 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5217262-da2d-4a92-8375-17f0138dd522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:03.193800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5217262-da2d-4a92-8375-17f0138dd522 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:03.194079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5217262-da2d-4a92-8375-17f0138dd522 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:03.194349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5217262-da2d-4a92-8375-17f0138dd522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:03.254802 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 518/1041] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:20:03.265221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5217262-da2d-4a92-8375-17f0138dd522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:03.266930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5217262-da2d-4a92-8375-17f0138dd522 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:03.369080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 524/1042] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:20:03 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:20:03.374414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfaae4-c5d1-4376-b369-f0c0b31023fc None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:03.376998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:03.477498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfaae4-c5d1-4376-b369-f0c0b31023fc None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:03.512279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206c38c-b582-42e8-9246-2f05140913d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:03.512942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206c38c-b582-42e8-9246-2f05140913d7 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:03.513196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206c38c-b582-42e8-9246-2f05140913d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:03.513463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206c38c-b582-42e8-9246-2f05140913d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:03.523422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5217262-da2d-4a92-8375-17f0138dd522 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:03.524982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206c38c-b582-42e8-9246-2f05140913d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:03.526607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206c38c-b582-42e8-9246-2f05140913d7 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-e206c38c-b582-42e8-9246-2f05140913d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:20:03.545766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:03.546719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:20:03.547731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:20:03.547731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:03.547731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:03.547731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 519/1043] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:03.552642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206c38c-b582-42e8-9246-2f05140913d7 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:03.553502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:03.610900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:03.633244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5217262-da2d-4a92-8375-17f0138dd522 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:03.664716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206c38c-b582-42e8-9246-2f05140913d7 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:03.675639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024c5d5-e1b4-41c6-8559-a1dc79590482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:03.676516 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024c5d5-e1b4-41c6-8559-a1dc79590482 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:03.676820 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024c5d5-e1b4-41c6-8559-a1dc79590482 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:03.677096 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024c5d5-e1b4-41c6-8559-a1dc79590482 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:03.690905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024c5d5-e1b4-41c6-8559-a1dc79590482 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-3024c5d5-e1b4-41c6-8559-a1dc79590482 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:20:03.695528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:03.696443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:03.697345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 525/1044] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:20:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:20:03.702310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024c5d5-e1b4-41c6-8559-a1dc79590482 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:03.706565 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10290777-78ee-4dac-97f8-0e4520dc2751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:03.707180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10290777-78ee-4dac-97f8-0e4520dc2751 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:03.707456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10290777-78ee-4dac-97f8-0e4520dc2751 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:03.707724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10290777-78ee-4dac-97f8-0e4520dc2751 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:03.708366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10290777-78ee-4dac-97f8-0e4520dc2751 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:03.708664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10290777-78ee-4dac-97f8-0e4520dc2751 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:03.710042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10290777-78ee-4dac-97f8-0e4520dc2751 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:03.725042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 520/1045] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:03.728723 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10290777-78ee-4dac-97f8-0e4520dc2751 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:03.806360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:03.821038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024c5d5-e1b4-41c6-8559-a1dc79590482 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:03.834969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10290777-78ee-4dac-97f8-0e4520dc2751 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:03.848986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:03.860696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192214e5-c005-47aa-923f-87a79f436fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:03.861618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192214e5-c005-47aa-923f-87a79f436fde None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:03.861901 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192214e5-c005-47aa-923f-87a79f436fde None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:03.862694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192214e5-c005-47aa-923f-87a79f436fde None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:03.863625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192214e5-c005-47aa-923f-87a79f436fde None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:03.863932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192214e5-c005-47aa-923f-87a79f436fde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:03.865987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192214e5-c005-47aa-923f-87a79f436fde None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:03.870421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffb0f5-023a-4ef6-b569-4565de8865bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:03.871056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffb0f5-023a-4ef6-b569-4565de8865bc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:03.871338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffb0f5-023a-4ef6-b569-4565de8865bc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:03.872023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffb0f5-023a-4ef6-b569-4565de8865bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:03.874885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 526/1046] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:20:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:03.880137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffb0f5-023a-4ef6-b569-4565de8865bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:03.880989 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192214e5-c005-47aa-923f-87a79f436fde None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:03.881585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffb0f5-023a-4ef6-b569-4565de8865bc None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-0dffb0f5-023a-4ef6-b569-4565de8865bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:20:03.894403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:03.897067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:03.898055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:20:03.898055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:20:03.898055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:20:03.898055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:20:03.898055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:20:03.898055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:20:03.898055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:20:03.898055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:03.898055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 521/1047] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:20:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:03.900959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffb0f5-023a-4ef6-b569-4565de8865bc None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:03.992666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192214e5-c005-47aa-923f-87a79f436fde None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:04.004663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffb0f5-023a-4ef6-b569-4565de8865bc None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:04.029927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2deb73-b10c-4053-8be4-29b9a10c56c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:04.030557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2deb73-b10c-4053-8be4-29b9a10c56c2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:04.030947 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2deb73-b10c-4053-8be4-29b9a10c56c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:04.031220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2deb73-b10c-4053-8be4-29b9a10c56c2 None admin] PATH_INFO: `/v3/users/be7174167b4a4ff8894e24018f5f8cb7` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:04.039608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec41110-2a3d-4581-9e23-15d653bb4159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:04.040216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec41110-2a3d-4581-9e23-15d653bb4159 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:04.040498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec41110-2a3d-4581-9e23-15d653bb4159 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:04.040791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec41110-2a3d-4581-9e23-15d653bb4159 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:04.041493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec41110-2a3d-4581-9e23-15d653bb4159 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:04.041805 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec41110-2a3d-4581-9e23-15d653bb4159 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:04.043232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec41110-2a3d-4581-9e23-15d653bb4159 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:04.046598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2deb73-b10c-4053-8be4-29b9a10c56c2 None admin] RBAC: Authorizing `identity:get_user(user_id=be7174167b4a4ff8894e24018f5f8cb7)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:04.048205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2deb73-b10c-4053-8be4-29b9a10c56c2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:04.050407 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 527/1048] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:03 2022] GET /identity/v3/users/be7174167b4a4ff8894e24018f5f8cb7 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:04.055930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2deb73-b10c-4053-8be4-29b9a10c56c2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:04.060794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 522/1049] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:20:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:04.066988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec41110-2a3d-4581-9e23-15d653bb4159 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:04.071580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:04.104824 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:04.166454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2deb73-b10c-4053-8be4-29b9a10c56c2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:04.173606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec41110-2a3d-4581-9e23-15d653bb4159 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:04.202886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610fb6e-fd9e-463f-982f-d1df97319344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:04.203524 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610fb6e-fd9e-463f-982f-d1df97319344 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:04.203801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610fb6e-fd9e-463f-982f-d1df97319344 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:04.204070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610fb6e-fd9e-463f-982f-d1df97319344 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:04.210126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944bd8-fbb0-41af-87d8-f8927ea7c78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:04.210731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944bd8-fbb0-41af-87d8-f8927ea7c78a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:04.211012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944bd8-fbb0-41af-87d8-f8927ea7c78a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:04.211287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944bd8-fbb0-41af-87d8-f8927ea7c78a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:04.212012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944bd8-fbb0-41af-87d8-f8927ea7c78a None admin] RBAC: Adding query filter params (role.id=2dc5eb239bdc4aa6be297199d70bb634, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=4739e80619a54b178b9aacec6a42a7f0) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:04.212345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944bd8-fbb0-41af-87d8-f8927ea7c78a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:04.213787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944bd8-fbb0-41af-87d8-f8927ea7c78a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:04.228788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 523/1050] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:20:04 2022] GET /identity/v3/role_assignments?user.id=4739e80619a54b178b9aacec6a42a7f0&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=2dc5eb239bdc4aa6be297199d70bb634 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:04.289312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610fb6e-fd9e-463f-982f-d1df97319344 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:04.290774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610fb6e-fd9e-463f-982f-d1df97319344 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:04.303010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:04.402646 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 528/1051] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:20:04 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 16:20:04.408657 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944bd8-fbb0-41af-87d8-f8927ea7c78a None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:04.513907 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:04.518231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944bd8-fbb0-41af-87d8-f8927ea7c78a None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:04.553327 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854290e-6577-480f-91d0-c03a18ca594a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:04.553957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854290e-6577-480f-91d0-c03a18ca594a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:04.554234 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854290e-6577-480f-91d0-c03a18ca594a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:04.554505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854290e-6577-480f-91d0-c03a18ca594a None admin] PATH_INFO: `/v3/projects/158c81a6e0a84b05b14281c3abafce78` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:04.555623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854290e-6577-480f-91d0-c03a18ca594a None admin] RBAC: Authorizing `identity:get_project(project_id=158c81a6e0a84b05b14281c3abafce78)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:04.557076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854290e-6577-480f-91d0-c03a18ca594a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:04.558864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 524/1052] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:20:04 2022] GET /identity/v3/projects/158c81a6e0a84b05b14281c3abafce78 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:04.564698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610fb6e-fd9e-463f-982f-d1df97319344 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:04.582362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06091ef-da2e-4ad1-bbd4-a97a48c495cf None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:04.582662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06091ef-da2e-4ad1-bbd4-a97a48c495cf None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:04.582911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06091ef-da2e-4ad1-bbd4-a97a48c495cf None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:04.584222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 525/1053] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:20:04.589104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23c4c5-ede7-4203-a3e3-7b7671a9eee9 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:04.589387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23c4c5-ede7-4203-a3e3-7b7671a9eee9 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:04.589829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23c4c5-ede7-4203-a3e3-7b7671a9eee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:04.608198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ca23c4c5-ede7-4203-a3e3-7b7671a9eee9 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:04.609403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ca23c4c5-ede7-4203-a3e3-7b7671a9eee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:04.676695 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610fb6e-fd9e-463f-982f-d1df97319344 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:04.685108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 526/1054] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:20:04.686852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:04.712803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad8dbc-3650-422f-8233-783ea327e014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:04.713411 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad8dbc-3650-422f-8233-783ea327e014 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:04.713690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad8dbc-3650-422f-8233-783ea327e014 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:04.714019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad8dbc-3650-422f-8233-783ea327e014 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:04.714755 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad8dbc-3650-422f-8233-783ea327e014 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=158c81a6e0a84b05b14281c3abafce78, user.id=be7174167b4a4ff8894e24018f5f8cb7) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:04.715063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad8dbc-3650-422f-8233-783ea327e014 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:04.716490 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad8dbc-3650-422f-8233-783ea327e014 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:04.729003 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 529/1055] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:20:04 2022] GET /identity/v3/role_assignments?user.id=be7174167b4a4ff8894e24018f5f8cb7&scope.project.id=158c81a6e0a84b05b14281c3abafce78&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:04.958393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23c4c5-ede7-4203-a3e3-7b7671a9eee9 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:05.060478 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:05.068064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23c4c5-ede7-4203-a3e3-7b7671a9eee9 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:05.103577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b3c2e-3786-4371-ba1f-eb0bf37cf11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:05.104206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b3c2e-3786-4371-ba1f-eb0bf37cf11f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:05.104480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b3c2e-3786-4371-ba1f-eb0bf37cf11f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:05.104832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b3c2e-3786-4371-ba1f-eb0bf37cf11f None admin] PATH_INFO: `/v3/services/image` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:05.105467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b3c2e-3786-4371-ba1f-eb0bf37cf11f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:05.106868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b3c2e-3786-4371-ba1f-eb0bf37cf11f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-194b3c2e-3786-4371-ba1f-eb0bf37cf11f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:05.115570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:05.116643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:05.121861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 527/1056] 10.222.0.55 () {56 vars in 1105 bytes} [Fri Aug 19 16:20:04 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:05.122725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad8dbc-3650-422f-8233-783ea327e014 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:05.228854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad8dbc-3650-422f-8233-783ea327e014 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:05.261896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a95f95-1490-4d18-9235-3e7956077c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:05.262508 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a95f95-1490-4d18-9235-3e7956077c4e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:05.262788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a95f95-1490-4d18-9235-3e7956077c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:05.263062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a95f95-1490-4d18-9235-3e7956077c4e None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:05.263685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a95f95-1490-4d18-9235-3e7956077c4e None admin] RBAC: Adding query filter params (name=image) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:05.263985 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a95f95-1490-4d18-9235-3e7956077c4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:05.265388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a95f95-1490-4d18-9235-3e7956077c4e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:05.270813 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 530/1057] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:05.276326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b3c2e-3786-4371-ba1f-eb0bf37cf11f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:05.378798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b3c2e-3786-4371-ba1f-eb0bf37cf11f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:05.411862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12df21f-16e6-4c70-ad9d-cf1b189c9e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:05.412461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12df21f-16e6-4c70-ad9d-cf1b189c9e6f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:05.412735 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12df21f-16e6-4c70-ad9d-cf1b189c9e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:05.413028 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12df21f-16e6-4c70-ad9d-cf1b189c9e6f None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:05.413642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12df21f-16e6-4c70-ad9d-cf1b189c9e6f None admin] RBAC: Adding query filter params (type=image) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:05.413937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12df21f-16e6-4c70-ad9d-cf1b189c9e6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:05.415280 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12df21f-16e6-4c70-ad9d-cf1b189c9e6f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:05.424656 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 528/1058] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:05.425997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a95f95-1490-4d18-9235-3e7956077c4e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:05.462052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:05.527612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a95f95-1490-4d18-9235-3e7956077c4e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:05.560554 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e35ec-7617-4170-aa2a-3eb8e0c2e3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:05.561188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e35ec-7617-4170-aa2a-3eb8e0c2e3cb None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:05.561467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e35ec-7617-4170-aa2a-3eb8e0c2e3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:05.561736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e35ec-7617-4170-aa2a-3eb8e0c2e3cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:05.562397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e35ec-7617-4170-aa2a-3eb8e0c2e3cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:05.563762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e35ec-7617-4170-aa2a-3eb8e0c2e3cb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:05.586333 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 531/1059] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:20:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:20:06.134217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ce49d-e0ff-43e0-b6ba-3b89cfd0b8e6 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:06.134767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ce49d-e0ff-43e0-b6ba-3b89cfd0b8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:06.134767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ce49d-e0ff-43e0-b6ba-3b89cfd0b8e6 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:06.136104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 529/1060] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:06.141432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6191bfb-1a97-43b2-8529-aa8669e651c9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:06.141731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6191bfb-1a97-43b2-8529-aa8669e651c9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:06.142010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6191bfb-1a97-43b2-8529-aa8669e651c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:06.160742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a6191bfb-1a97-43b2-8529-aa8669e651c9 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:06.161982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a6191bfb-1a97-43b2-8529-aa8669e651c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:06.239184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 532/1061] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:06.507308 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ce49d-e0ff-43e0-b6ba-3b89cfd0b8e6 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:06.536975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:06.611831 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6eb4f2-a368-4f6f-9839-c04a850837fa None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:06.612910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6eb4f2-a368-4f6f-9839-c04a850837fa None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:06.613004 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6eb4f2-a368-4f6f-9839-c04a850837fa None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:06.614266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 533/1062] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:20:06.619200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97187fc3-a446-4557-9e99-e1c834e19e82 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:06.619439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97187fc3-a446-4557-9e99-e1c834e19e82 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:06.619716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97187fc3-a446-4557-9e99-e1c834e19e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:06.624085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ce49d-e0ff-43e0-b6ba-3b89cfd0b8e6 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:06.639461 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-97187fc3-a446-4557-9e99-e1c834e19e82 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:06.640993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-97187fc3-a446-4557-9e99-e1c834e19e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:06.659217 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:06.667499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a835-0f16-48a3-b55b-526dfe472944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:06.668147 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a835-0f16-48a3-b55b-526dfe472944 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:06.668424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a835-0f16-48a3-b55b-526dfe472944 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:06.668694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a835-0f16-48a3-b55b-526dfe472944 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:06.681019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1a835-0f16-48a3-b55b-526dfe472944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-24d1a835-0f16-48a3-b55b-526dfe472944 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:20:06.685928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:06.686949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:06.687791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:06.687791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:06.687791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:06.687791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:06.687791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:20:06.687791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:20:06.687791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:20:06.687791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:20:06.687791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:20:06.687791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:20:06.687791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 16:20:06.687791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:06.693219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 530/1063] 10.222.0.55 () {56 vars in 1120 bytes} [Fri Aug 19 16:20:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:06.694337 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a835-0f16-48a3-b55b-526dfe472944 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:06.723834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 534/1064] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:20:06.782998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:06.799961 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a835-0f16-48a3-b55b-526dfe472944 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:06.834776 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6162a-e9e9-4e0a-beff-6d9b9e2f0e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:06.835378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6162a-e9e9-4e0a-beff-6d9b9e2f0e8c None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:06.835658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6162a-e9e9-4e0a-beff-6d9b9e2f0e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:06.835932 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6162a-e9e9-4e0a-beff-6d9b9e2f0e8c None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:06.836586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6162a-e9e9-4e0a-beff-6d9b9e2f0e8c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:06.836916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6162a-e9e9-4e0a-beff-6d9b9e2f0e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:06.838275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6162a-e9e9-4e0a-beff-6d9b9e2f0e8c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:06.847223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 531/1065] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Aug 19 16:20:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:06.853240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97187fc3-a446-4557-9e99-e1c834e19e82 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:06.953356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97187fc3-a446-4557-9e99-e1c834e19e82 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:06.987938 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d40cf-a2dd-4039-b59a-b7d4a5ea8fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:06.988580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d40cf-a2dd-4039-b59a-b7d4a5ea8fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:06.988890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d40cf-a2dd-4039-b59a-b7d4a5ea8fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:06.989170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d40cf-a2dd-4039-b59a-b7d4a5ea8fd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:06.998256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6162a-e9e9-4e0a-beff-6d9b9e2f0e8c None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:06.999507 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d40cf-a2dd-4039-b59a-b7d4a5ea8fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:07.000994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d40cf-a2dd-4039-b59a-b7d4a5ea8fd4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:07.011797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-654d40cf-a2dd-4039-b59a-b7d4a5ea8fd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:20:07.019204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:07.020176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:20:07.021091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:20:07.021091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:07.021091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:07.022487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 535/1066] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:07.031198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d40cf-a2dd-4039-b59a-b7d4a5ea8fd4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:07.110733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6162a-e9e9-4e0a-beff-6d9b9e2f0e8c None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:07.139947 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d40cf-a2dd-4039-b59a-b7d4a5ea8fd4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:07.145761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd419b-d20c-4ed8-92f4-3f2087d56aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:07.146392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd419b-d20c-4ed8-92f4-3f2087d56aae None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:07.146666 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd419b-d20c-4ed8-92f4-3f2087d56aae None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:07.146933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd419b-d20c-4ed8-92f4-3f2087d56aae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:07.158260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd419b-d20c-4ed8-92f4-3f2087d56aae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-06fd419b-d20c-4ed8-92f4-3f2087d56aae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:20:07.163435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:07.164333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:07.165214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 532/1067] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:20:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:20:07.170608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd419b-d20c-4ed8-92f4-3f2087d56aae None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:07.179927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35766cf5-ff3f-4d52-aae1-8817722c0073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:07.180547 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35766cf5-ff3f-4d52-aae1-8817722c0073 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:07.180884 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35766cf5-ff3f-4d52-aae1-8817722c0073 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:07.181163 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35766cf5-ff3f-4d52-aae1-8817722c0073 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:07.181294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:07.181816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35766cf5-ff3f-4d52-aae1-8817722c0073 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:07.182119 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35766cf5-ff3f-4d52-aae1-8817722c0073 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:07.183604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35766cf5-ff3f-4d52-aae1-8817722c0073 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:07.196648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 536/1068] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:07.202312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35766cf5-ff3f-4d52-aae1-8817722c0073 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:07.266248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:07.281077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd419b-d20c-4ed8-92f4-3f2087d56aae None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:07.314113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35766cf5-ff3f-4d52-aae1-8817722c0073 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:07.317993 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3dbfbd-132d-486a-9c3c-81027e671733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:07.318603 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dbfbd-132d-486a-9c3c-81027e671733 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:07.318881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dbfbd-132d-486a-9c3c-81027e671733 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:07.319151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3dbfbd-132d-486a-9c3c-81027e671733 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:07.319874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3dbfbd-132d-486a-9c3c-81027e671733 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:07.320176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3dbfbd-132d-486a-9c3c-81027e671733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:07.321764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3dbfbd-132d-486a-9c3c-81027e671733 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:07.336263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 533/1069] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:20:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:07.337130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3dbfbd-132d-486a-9c3c-81027e671733 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:07.349595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d9621-0871-4d9c-8385-f071f27e8f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:07.350234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d9621-0871-4d9c-8385-f071f27e8f27 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:07.350518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d9621-0871-4d9c-8385-f071f27e8f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:07.350803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d9621-0871-4d9c-8385-f071f27e8f27 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:07.359423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d9621-0871-4d9c-8385-f071f27e8f27 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:07.360950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d9621-0871-4d9c-8385-f071f27e8f27 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-cb6d9621-0871-4d9c-8385-f071f27e8f27 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:20:07.372678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:07.373660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:07.374651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:07.374651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:07.374651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:20:07.374651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:20:07.374651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:20:07.374651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:20:07.374651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:20:07.374651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:20:07.374651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:20:07.374651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:07.374651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 537/1070] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Aug 19 16:20:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:07.379590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d9621-0871-4d9c-8385-f071f27e8f27 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:07.414355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:07.451223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3dbfbd-132d-486a-9c3c-81027e671733 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:07.488205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c38e9b-bf20-45b8-acfd-02aae3fab281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:07.488862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c38e9b-bf20-45b8-acfd-02aae3fab281 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:07.489144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c38e9b-bf20-45b8-acfd-02aae3fab281 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:07.489415 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c38e9b-bf20-45b8-acfd-02aae3fab281 None admin] PATH_INFO: `/v3/users/be7174167b4a4ff8894e24018f5f8cb7` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:07.489781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d9621-0871-4d9c-8385-f071f27e8f27 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:07.490578 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c38e9b-bf20-45b8-acfd-02aae3fab281 None admin] RBAC: Authorizing `identity:get_user(user_id=be7174167b4a4ff8894e24018f5f8cb7)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:07.492002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c38e9b-bf20-45b8-acfd-02aae3fab281 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:07.493622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 534/1071] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:07 2022] GET /identity/v3/users/be7174167b4a4ff8894e24018f5f8cb7 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:07.499332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c38e9b-bf20-45b8-acfd-02aae3fab281 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:07.512722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:07.528842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e64fd-76d1-4a71-81eb-e66dc5b91bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:07.529454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e64fd-76d1-4a71-81eb-e66dc5b91bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:07.529817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e64fd-76d1-4a71-81eb-e66dc5b91bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:07.530003 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e64fd-76d1-4a71-81eb-e66dc5b91bb7 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:07.530730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e64fd-76d1-4a71-81eb-e66dc5b91bb7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:07.531034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e64fd-76d1-4a71-81eb-e66dc5b91bb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:07.532540 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e64fd-76d1-4a71-81eb-e66dc5b91bb7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:07.554433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 538/1072] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Aug 19 16:20:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:07.554433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1afa6-6ba6-4ba8-85de-91fb8cc2292c None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:07.554433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1afa6-6ba6-4ba8-85de-91fb8cc2292c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:07.554433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1afa6-6ba6-4ba8-85de-91fb8cc2292c None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:07.554433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 539/1073] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:07.555325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1afa6-6ba6-4ba8-85de-91fb8cc2292c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:07.603993 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c38e9b-bf20-45b8-acfd-02aae3fab281 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:07.637242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991894c7-bf04-4918-af1f-05f32f8ba96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:07.637801 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991894c7-bf04-4918-af1f-05f32f8ba96f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:07.638073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991894c7-bf04-4918-af1f-05f32f8ba96f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:07.638368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991894c7-bf04-4918-af1f-05f32f8ba96f None admin] PATH_INFO: `/v3/projects/158c81a6e0a84b05b14281c3abafce78` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:07.641618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991894c7-bf04-4918-af1f-05f32f8ba96f None admin] RBAC: Authorizing `identity:get_project(project_id=158c81a6e0a84b05b14281c3abafce78)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:07.641618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991894c7-bf04-4918-af1f-05f32f8ba96f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:07.642683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 535/1074] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:20:07 2022] GET /identity/v3/projects/158c81a6e0a84b05b14281c3abafce78 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:07.644619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2e838-dad5-43f3-8fe6-02ee46cdb327 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:07.644923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2e838-dad5-43f3-8fe6-02ee46cdb327 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:07.645195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2e838-dad5-43f3-8fe6-02ee46cdb327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:07.654202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:07.662684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1afa6-6ba6-4ba8-85de-91fb8cc2292c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:07.665338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-96e2e838-dad5-43f3-8fe6-02ee46cdb327 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:07.666513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-96e2e838-dad5-43f3-8fe6-02ee46cdb327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:07.697053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1728a-8fbb-4a27-9620-6d29e89ad8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:07.697680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1728a-8fbb-4a27-9620-6d29e89ad8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:07.697955 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1728a-8fbb-4a27-9620-6d29e89ad8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:07.698225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1728a-8fbb-4a27-9620-6d29e89ad8a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:07.709453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1728a-8fbb-4a27-9620-6d29e89ad8a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:07.710916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1728a-8fbb-4a27-9620-6d29e89ad8a6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-ccc1728a-8fbb-4a27-9620-6d29e89ad8a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:20:07.725031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:07.726379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:20:07.727282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:20:07.727282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:07.727282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:07.727282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 540/1075] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:20:07.727513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1728a-8fbb-4a27-9620-6d29e89ad8a6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:07.735731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 536/1076] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:07.736545 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2e838-dad5-43f3-8fe6-02ee46cdb327 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:07.760932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:07.830977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1728a-8fbb-4a27-9620-6d29e89ad8a6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:07.839534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2e838-dad5-43f3-8fe6-02ee46cdb327 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:07.867570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34aae8-6b30-4a96-b023-43eb7ca5605f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:07.868508 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34aae8-6b30-4a96-b023-43eb7ca5605f None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:07.868980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34aae8-6b30-4a96-b023-43eb7ca5605f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:07.869259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34aae8-6b30-4a96-b023-43eb7ca5605f None admin] PATH_INFO: `/v3/projects/158c81a6e0a84b05b14281c3abafce78/users/be7174167b4a4ff8894e24018f5f8cb7/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:07.871184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34aae8-6b30-4a96-b023-43eb7ca5605f None admin] RBAC: Authorizing `identity:create_grant(project_id=158c81a6e0a84b05b14281c3abafce78, user_id=be7174167b4a4ff8894e24018f5f8cb7, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:07.872610 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34aae8-6b30-4a96-b023-43eb7ca5605f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:07.876201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f3022-2851-4f96-9a4c-279483903b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:07.877483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f3022-2851-4f96-9a4c-279483903b0f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:07.877959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f3022-2851-4f96-9a4c-279483903b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:07.878428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f3022-2851-4f96-9a4c-279483903b0f None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:07.879604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f3022-2851-4f96-9a4c-279483903b0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:07.885026 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f3022-2851-4f96-9a4c-279483903b0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:07.885026 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f3022-2851-4f96-9a4c-279483903b0f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:07.891787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:07.896865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 541/1077] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:20:07 2022] PUT /identity/v3/projects/158c81a6e0a84b05b14281c3abafce78/users/be7174167b4a4ff8894e24018f5f8cb7/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:20:07.897518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 537/1078] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:07.903166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f3022-2851-4f96-9a4c-279483903b0f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:08.009249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34aae8-6b30-4a96-b023-43eb7ca5605f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:08.014531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f3022-2851-4f96-9a4c-279483903b0f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:08.056498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d72d1-876d-4c5d-949c-21c9062d8c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:08.057165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d72d1-876d-4c5d-949c-21c9062d8c61 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:08.057448 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d72d1-876d-4c5d-949c-21c9062d8c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:08.057723 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d72d1-876d-4c5d-949c-21c9062d8c61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:08.064432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d72d1-876d-4c5d-949c-21c9062d8c61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:08.066078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d72d1-876d-4c5d-949c-21c9062d8c61 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-796d72d1-876d-4c5d-949c-21c9062d8c61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:20:08.078631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:08.079556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:08.081156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:20:08.081156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:20:08.081156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:20:08.081156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:20:08.081156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:20:08.081156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:20:08.081156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:20:08.081156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:08.081156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 538/1079] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:20:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:08.085912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d72d1-876d-4c5d-949c-21c9062d8c61 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:08.124733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:08.138164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34aae8-6b30-4a96-b023-43eb7ca5605f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:08.141505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:08.176896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab9359-fe34-43af-841d-821de149e278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:08.177859 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab9359-fe34-43af-841d-821de149e278 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:08.178360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab9359-fe34-43af-841d-821de149e278 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:08.178779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab9359-fe34-43af-841d-821de149e278 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:08.186798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d72d1-876d-4c5d-949c-21c9062d8c61 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:08.221922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673a3ed-0e11-4f33-8841-fd56f0b4c0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:08.222530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673a3ed-0e11-4f33-8841-fd56f0b4c0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:08.222808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673a3ed-0e11-4f33-8841-fd56f0b4c0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:08.223077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673a3ed-0e11-4f33-8841-fd56f0b4c0e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:08.223762 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673a3ed-0e11-4f33-8841-fd56f0b4c0e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:08.224062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673a3ed-0e11-4f33-8841-fd56f0b4c0e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:08.225549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673a3ed-0e11-4f33-8841-fd56f0b4c0e9 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:08.237885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 539/1080] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:20:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:08.243659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673a3ed-0e11-4f33-8841-fd56f0b4c0e9 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:08.253663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab9359-fe34-43af-841d-821de149e278 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:08.255483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab9359-fe34-43af-841d-821de149e278 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:08.348602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:08.354987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673a3ed-0e11-4f33-8841-fd56f0b4c0e9 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:08.365207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 542/1081] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:20:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 16:20:08.367237 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab9359-fe34-43af-841d-821de149e278 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:08.372516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:08.395265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c527d4-96ec-44b2-9567-d2a82cd86d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:08.395928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c527d4-96ec-44b2-9567-d2a82cd86d68 None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:08.396203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c527d4-96ec-44b2-9567-d2a82cd86d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:08.396476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c527d4-96ec-44b2-9567-d2a82cd86d68 None admin] PATH_INFO: `/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/4739e80619a54b178b9aacec6a42a7f0/roles/2dc5eb239bdc4aa6be297199d70bb634` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:08.398709 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c527d4-96ec-44b2-9567-d2a82cd86d68 None admin] RBAC: Authorizing `identity:create_grant(project_id=d143cf444f354ec0bf2089dae3d94255, user_id=4739e80619a54b178b9aacec6a42a7f0, role_id=2dc5eb239bdc4aa6be297199d70bb634)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:08.400128 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c527d4-96ec-44b2-9567-d2a82cd86d68 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:08.413692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 540/1082] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:20:08 2022] PUT /identity/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/4739e80619a54b178b9aacec6a42a7f0/roles/2dc5eb239bdc4aa6be297199d70bb634 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:20:08.478798 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab9359-fe34-43af-841d-821de149e278 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:08.517568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72827ba5-912f-49f6-8cb4-ce3dd188d985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:08.518789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72827ba5-912f-49f6-8cb4-ce3dd188d985 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:08.519245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72827ba5-912f-49f6-8cb4-ce3dd188d985 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:08.519603 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72827ba5-912f-49f6-8cb4-ce3dd188d985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:08.531206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72827ba5-912f-49f6-8cb4-ce3dd188d985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:08.532681 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72827ba5-912f-49f6-8cb4-ce3dd188d985 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-72827ba5-912f-49f6-8cb4-ce3dd188d985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:20:08.550086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:08.551432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:20:08.553234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:20:08.553234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:08.553234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:08.553234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 543/1083] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:20:08.557121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c527d4-96ec-44b2-9567-d2a82cd86d68 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:08.667291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c527d4-96ec-44b2-9567-d2a82cd86d68 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:08.703366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe88a3-8776-4de2-a1e3-30d1b180f3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:08.703976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe88a3-8776-4de2-a1e3-30d1b180f3ec None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:08.704272 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe88a3-8776-4de2-a1e3-30d1b180f3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:08.704532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe88a3-8776-4de2-a1e3-30d1b180f3ec None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:08.705202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe88a3-8776-4de2-a1e3-30d1b180f3ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:08.705533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe88a3-8776-4de2-a1e3-30d1b180f3ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:08.706949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe88a3-8776-4de2-a1e3-30d1b180f3ec None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:08.718834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 541/1084] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:09.795787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ff642-2057-4265-b880-07bf99db5277 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:09.796330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ff642-2057-4265-b880-07bf99db5277 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:09.796421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ff642-2057-4265-b880-07bf99db5277 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:09.797816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 544/1085] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:09.803308 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a26c30-5418-4a48-8355-40e077381554 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:09.803705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a26c30-5418-4a48-8355-40e077381554 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:09.803926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a26c30-5418-4a48-8355-40e077381554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:09.822347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d6a26c30-5418-4a48-8355-40e077381554 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:09.823515 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d6a26c30-5418-4a48-8355-40e077381554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:09.849420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:09.892268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 542/1086] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:10.164068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ff642-2057-4265-b880-07bf99db5277 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:10.208561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:10.267395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ff642-2057-4265-b880-07bf99db5277 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:10.300438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e196c06-8725-406d-b592-c723eefbe4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:10.301094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e196c06-8725-406d-b592-c723eefbe4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:10.301377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e196c06-8725-406d-b592-c723eefbe4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:10.301649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e196c06-8725-406d-b592-c723eefbe4f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:10.311993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e196c06-8725-406d-b592-c723eefbe4f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-3e196c06-8725-406d-b592-c723eefbe4f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:20:10.317307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:10.318469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:10.319286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 545/1087] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:20:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:10.324421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a26c30-5418-4a48-8355-40e077381554 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:10.343019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb4870-f358-425d-b8aa-7b4b471805ee None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:10.343301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb4870-f358-425d-b8aa-7b4b471805ee None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:10.343572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb4870-f358-425d-b8aa-7b4b471805ee None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:10.344956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 546/1088] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:20:10.349883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc5a6e-6113-41fc-8688-a43aac8f2821 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:10.350171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc5a6e-6113-41fc-8688-a43aac8f2821 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:10.350449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc5a6e-6113-41fc-8688-a43aac8f2821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:10.368984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8bbc5a6e-6113-41fc-8688-a43aac8f2821 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:10.370199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8bbc5a6e-6113-41fc-8688-a43aac8f2821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:10.421772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a26c30-5418-4a48-8355-40e077381554 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:10.436859 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 547/1089] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:20:10.455787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b35c83-f54c-4282-8a05-517480034236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:10.456391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b35c83-f54c-4282-8a05-517480034236 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:10.456670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b35c83-f54c-4282-8a05-517480034236 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:10.456973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b35c83-f54c-4282-8a05-517480034236 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:10.457625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b35c83-f54c-4282-8a05-517480034236 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:10.457926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b35c83-f54c-4282-8a05-517480034236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:10.459282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b35c83-f54c-4282-8a05-517480034236 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:10.471845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 543/1090] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:20:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:10.472995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc5a6e-6113-41fc-8688-a43aac8f2821 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:10.504228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:10.571360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc5a6e-6113-41fc-8688-a43aac8f2821 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:10.605244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829bd29-057d-482b-b39d-4b75b6545377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:10.605877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829bd29-057d-482b-b39d-4b75b6545377 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:10.606159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829bd29-057d-482b-b39d-4b75b6545377 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:10.606437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829bd29-057d-482b-b39d-4b75b6545377 None admin] PATH_INFO: `/v3/users/be7174167b4a4ff8894e24018f5f8cb7` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:10.607568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829bd29-057d-482b-b39d-4b75b6545377 None admin] RBAC: Authorizing `identity:get_user(user_id=be7174167b4a4ff8894e24018f5f8cb7)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:10.609033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829bd29-057d-482b-b39d-4b75b6545377 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:10.610604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 548/1091] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:10 2022] GET /identity/v3/users/be7174167b4a4ff8894e24018f5f8cb7 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:10.617126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b35c83-f54c-4282-8a05-517480034236 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:10.644624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:10.709722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829bd29-057d-482b-b39d-4b75b6545377 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:10.715936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b35c83-f54c-4282-8a05-517480034236 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:10.750163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7276b4-6b9e-4c01-a683-18729a976033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:10.750764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7276b4-6b9e-4c01-a683-18729a976033 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:10.751041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7276b4-6b9e-4c01-a683-18729a976033 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:10.751305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7276b4-6b9e-4c01-a683-18729a976033 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:10.808337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:10.811395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829bd29-057d-482b-b39d-4b75b6545377 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:10.816500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7276b4-6b9e-4c01-a683-18729a976033 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:10.818050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7276b4-6b9e-4c01-a683-18729a976033 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:10.843639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:10.844772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bb005-d9dd-496d-9d13-3ddbcf2df9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:10.845395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb005-d9dd-496d-9d13-3ddbcf2df9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:10.845676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb005-d9dd-496d-9d13-3ddbcf2df9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:10.846532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb005-d9dd-496d-9d13-3ddbcf2df9c0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:10.856550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bb005-d9dd-496d-9d13-3ddbcf2df9c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-7c7bb005-d9dd-496d-9d13-3ddbcf2df9c0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:20:10.861226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:10.862854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:10.863680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 549/1092] 10.222.0.55 () {56 vars in 1120 bytes} [Fri Aug 19 16:20:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:20:10.868772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bb005-d9dd-496d-9d13-3ddbcf2df9c0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:10.922071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 544/1093] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:20:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:20:10.928605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7276b4-6b9e-4c01-a683-18729a976033 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:10.966210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bb005-d9dd-496d-9d13-3ddbcf2df9c0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:11.001262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f65573-113d-4e96-a6df-891924518428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:11.001866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f65573-113d-4e96-a6df-891924518428 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:11.002144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f65573-113d-4e96-a6df-891924518428 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:11.002422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f65573-113d-4e96-a6df-891924518428 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:11.003086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f65573-113d-4e96-a6df-891924518428 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:11.003386 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f65573-113d-4e96-a6df-891924518428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:11.004781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f65573-113d-4e96-a6df-891924518428 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:11.012599 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 550/1094] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Aug 19 16:20:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:11.018719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f65573-113d-4e96-a6df-891924518428 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:11.026283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7276b4-6b9e-4c01-a683-18729a976033 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:11.043011 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:11.057413 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:11.068190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48d977-3fa1-4467-9e11-306e5ff63379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:11.068844 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48d977-3fa1-4467-9e11-306e5ff63379 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:11.069123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48d977-3fa1-4467-9e11-306e5ff63379 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:11.069394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48d977-3fa1-4467-9e11-306e5ff63379 None admin] PATH_INFO: `/v3/projects/158c81a6e0a84b05b14281c3abafce78` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:11.070547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48d977-3fa1-4467-9e11-306e5ff63379 None admin] RBAC: Authorizing `identity:get_project(project_id=158c81a6e0a84b05b14281c3abafce78)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:11.072104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48d977-3fa1-4467-9e11-306e5ff63379 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:11.073899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 545/1095] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:20:10 2022] GET /identity/v3/projects/158c81a6e0a84b05b14281c3abafce78 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:11.083233 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48d977-3fa1-4467-9e11-306e5ff63379 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:11.124993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f65573-113d-4e96-a6df-891924518428 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:11.157685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aabbee-5f69-45ca-84be-1f1f18e0a4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:11.158314 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aabbee-5f69-45ca-84be-1f1f18e0a4ed None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:11.159235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aabbee-5f69-45ca-84be-1f1f18e0a4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:11.159582 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aabbee-5f69-45ca-84be-1f1f18e0a4ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:11.169812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aabbee-5f69-45ca-84be-1f1f18e0a4ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:11.171251 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aabbee-5f69-45ca-84be-1f1f18e0a4ed None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:11.181963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48d977-3fa1-4467-9e11-306e5ff63379 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-66aabbee-5f69-45ca-84be-1f1f18e0a4ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:20:11.184774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:11.185858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:20:11.187089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:20:11.187089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:11.187089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:11.187089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 551/1096] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:20:11.192139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aabbee-5f69-45ca-84be-1f1f18e0a4ed None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:11.216403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74592da5-74a1-435b-aab4-641707ef96e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:11.216958 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74592da5-74a1-435b-aab4-641707ef96e0 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:11.217237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74592da5-74a1-435b-aab4-641707ef96e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:11.217507 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74592da5-74a1-435b-aab4-641707ef96e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:11.218218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74592da5-74a1-435b-aab4-641707ef96e0 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=158c81a6e0a84b05b14281c3abafce78, user.id=be7174167b4a4ff8894e24018f5f8cb7) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:11.218523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74592da5-74a1-435b-aab4-641707ef96e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:11.219932 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74592da5-74a1-435b-aab4-641707ef96e0 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:11.227550 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 546/1097] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:20:11 2022] GET /identity/v3/role_assignments?user.id=be7174167b4a4ff8894e24018f5f8cb7&scope.project.id=158c81a6e0a84b05b14281c3abafce78&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:11.271158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:11.299046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aabbee-5f69-45ca-84be-1f1f18e0a4ed None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:11.334632 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7b729-3265-4afa-b908-4ec3e18c9529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:11.335253 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7b729-3265-4afa-b908-4ec3e18c9529 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:11.335531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7b729-3265-4afa-b908-4ec3e18c9529 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:11.335801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7b729-3265-4afa-b908-4ec3e18c9529 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:11.336449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7b729-3265-4afa-b908-4ec3e18c9529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:11.336770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7b729-3265-4afa-b908-4ec3e18c9529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:11.338158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7b729-3265-4afa-b908-4ec3e18c9529 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:11.351346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 552/1098] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:11.357285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74592da5-74a1-435b-aab4-641707ef96e0 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:11.419573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:11.461532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74592da5-74a1-435b-aab4-641707ef96e0 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:11.494649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ccbb-7247-459f-8cc1-28f7f69976e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:11.495280 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ccbb-7247-459f-8cc1-28f7f69976e0 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:11.495560 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ccbb-7247-459f-8cc1-28f7f69976e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:11.495831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ccbb-7247-459f-8cc1-28f7f69976e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:11.503767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3ccbb-7247-459f-8cc1-28f7f69976e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:11.505231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3ccbb-7247-459f-8cc1-28f7f69976e0 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-dfa3ccbb-7247-459f-8cc1-28f7f69976e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:20:11.515959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:11.517070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:11.518862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:11.518862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:11.518862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:20:11.518862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:20:11.518862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:20:11.518862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:20:11.518862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:20:11.518862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:20:11.518862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 16:20:11.518862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:11.521064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 547/1099] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Aug 19 16:20:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:20:11.523473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7b729-3265-4afa-b908-4ec3e18c9529 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:11.623420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7b729-3265-4afa-b908-4ec3e18c9529 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:11.658537 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4126d3e-1f12-424d-9049-7f5cb88a8ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:11.659406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4126d3e-1f12-424d-9049-7f5cb88a8ffb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:11.659694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4126d3e-1f12-424d-9049-7f5cb88a8ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:11.659971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4126d3e-1f12-424d-9049-7f5cb88a8ffb None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:11.660706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4126d3e-1f12-424d-9049-7f5cb88a8ffb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:11.661060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4126d3e-1f12-424d-9049-7f5cb88a8ffb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:11.662499 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4126d3e-1f12-424d-9049-7f5cb88a8ffb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:11.664193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:11.682238 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 553/1100] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Aug 19 16:20:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:11.688344 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ccbb-7247-459f-8cc1-28f7f69976e0 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:11.791365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ccbb-7247-459f-8cc1-28f7f69976e0 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:11.808617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:11.828399 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20eed58-3cd4-474f-98f2-1f6cac3077bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:11.829014 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20eed58-3cd4-474f-98f2-1f6cac3077bf None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:11.829291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20eed58-3cd4-474f-98f2-1f6cac3077bf None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:11.829559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20eed58-3cd4-474f-98f2-1f6cac3077bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:11.895242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20eed58-3cd4-474f-98f2-1f6cac3077bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:11.896680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20eed58-3cd4-474f-98f2-1f6cac3077bf None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:11.990338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 548/1101] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:20:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:20:11.996017 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4126d3e-1f12-424d-9049-7f5cb88a8ffb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:12.094516 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4126d3e-1f12-424d-9049-7f5cb88a8ffb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:12.130764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1a2d5-492a-4eb0-84e0-c6f1bee74f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:12.131388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1a2d5-492a-4eb0-84e0-c6f1bee74f42 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:12.131665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1a2d5-492a-4eb0-84e0-c6f1bee74f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:12.131934 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1a2d5-492a-4eb0-84e0-c6f1bee74f42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:12.141633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1a2d5-492a-4eb0-84e0-c6f1bee74f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:12.143060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1a2d5-492a-4eb0-84e0-c6f1bee74f42 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-fbe1a2d5-492a-4eb0-84e0-c6f1bee74f42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:20:12.157744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:12.159393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:20:12.160463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:20:12.160463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:20:12.160463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:12.160905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 554/1102] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:12.164918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20eed58-3cd4-474f-98f2-1f6cac3077bf None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:12.183704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:12.264137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20eed58-3cd4-474f-98f2-1f6cac3077bf None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:12.298864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec4033-a835-4593-8720-1d704d036e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:12.300078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec4033-a835-4593-8720-1d704d036e48 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:12.300391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec4033-a835-4593-8720-1d704d036e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:12.300663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec4033-a835-4593-8720-1d704d036e48 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:12.301324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec4033-a835-4593-8720-1d704d036e48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:12.301623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec4033-a835-4593-8720-1d704d036e48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:12.302981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec4033-a835-4593-8720-1d704d036e48 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:12.314087 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 549/1103] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:12.319810 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1a2d5-492a-4eb0-84e0-c6f1bee74f42 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:12.366471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:12.419948 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1a2d5-492a-4eb0-84e0-c6f1bee74f42 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:12.452497 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc624a1-8736-4bcc-a862-11846415a567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:12.453151 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc624a1-8736-4bcc-a862-11846415a567 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:12.453432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc624a1-8736-4bcc-a862-11846415a567 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:12.453703 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc624a1-8736-4bcc-a862-11846415a567 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:12.460777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc624a1-8736-4bcc-a862-11846415a567 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:12.462225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc624a1-8736-4bcc-a862-11846415a567 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-dcc624a1-8736-4bcc-a862-11846415a567 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:20:12.473661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:12.474510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:12.475369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:20:12.475369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:20:12.475369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:20:12.475369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:20:12.475369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:20:12.475369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:20:12.475369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:20:12.475369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:12.475369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 555/1104] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:20:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:12.480989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec4033-a835-4593-8720-1d704d036e48 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:12.557649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:12.578887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec4033-a835-4593-8720-1d704d036e48 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:12.612261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f93d6-a8c4-41bd-82b2-9ff611bd79c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:12.612901 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f93d6-a8c4-41bd-82b2-9ff611bd79c8 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:12.613179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f93d6-a8c4-41bd-82b2-9ff611bd79c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:12.613446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f93d6-a8c4-41bd-82b2-9ff611bd79c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:12.614119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f93d6-a8c4-41bd-82b2-9ff611bd79c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:12.614416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f93d6-a8c4-41bd-82b2-9ff611bd79c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:12.615834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f93d6-a8c4-41bd-82b2-9ff611bd79c8 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:12.627539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 550/1105] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:20:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:12.633471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc624a1-8736-4bcc-a862-11846415a567 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:12.733313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc624a1-8736-4bcc-a862-11846415a567 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:12.735598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:12.766562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ef741-52d5-438b-89be-4cec0840002f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:12.767159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ef741-52d5-438b-89be-4cec0840002f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:12.767437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ef741-52d5-438b-89be-4cec0840002f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:12.767706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ef741-52d5-438b-89be-4cec0840002f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:12.768977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ef741-52d5-438b-89be-4cec0840002f None admin] RBAC: Adding query filter params (role.id=2dc5eb239bdc4aa6be297199d70bb634, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=4739e80619a54b178b9aacec6a42a7f0) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:12.769281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ef741-52d5-438b-89be-4cec0840002f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:12.770698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ef741-52d5-438b-89be-4cec0840002f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:12.776413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 556/1106] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:20:12 2022] GET /identity/v3/role_assignments?user.id=4739e80619a54b178b9aacec6a42a7f0&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=2dc5eb239bdc4aa6be297199d70bb634 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:13.156886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abe82b-3f4a-430a-9f07-cf0b2e9de99c None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:13.157396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abe82b-3f4a-430a-9f07-cf0b2e9de99c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:13.157482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abe82b-3f4a-430a-9f07-cf0b2e9de99c None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:13.159007 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 551/1107] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:13.164279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6c681-5c79-4ed5-ba86-b07ce468ac0b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:13.164688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6c681-5c79-4ed5-ba86-b07ce468ac0b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:13.164918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6c681-5c79-4ed5-ba86-b07ce468ac0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:13.183654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f9a6c681-5c79-4ed5-ba86-b07ce468ac0b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:13.184965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f9a6c681-5c79-4ed5-ba86-b07ce468ac0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:13.254479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 557/1108] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:13.529975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abe82b-3f4a-430a-9f07-cf0b2e9de99c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:13.632373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abe82b-3f4a-430a-9f07-cf0b2e9de99c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:13.667737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c7024-413e-4be6-8116-b38f45a963fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:13.668364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c7024-413e-4be6-8116-b38f45a963fb None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:13.668644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c7024-413e-4be6-8116-b38f45a963fb None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:13.668948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c7024-413e-4be6-8116-b38f45a963fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:13.670061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c7024-413e-4be6-8116-b38f45a963fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:13.671488 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c7024-413e-4be6-8116-b38f45a963fb None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:13.673092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 552/1109] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:13.679586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6c681-5c79-4ed5-ba86-b07ce468ac0b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:13.775680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:13.779585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6c681-5c79-4ed5-ba86-b07ce468ac0b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:13.813321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdddb1-6d7f-4ebd-a2c9-61d87c9c6ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:13.813855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdddb1-6d7f-4ebd-a2c9-61d87c9c6ee8 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:13.814130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdddb1-6d7f-4ebd-a2c9-61d87c9c6ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:13.814400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdddb1-6d7f-4ebd-a2c9-61d87c9c6ee8 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:13.815083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdddb1-6d7f-4ebd-a2c9-61d87c9c6ee8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:13.816496 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdddb1-6d7f-4ebd-a2c9-61d87c9c6ee8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:13.836538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 558/1110] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:20:13 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:20:15.737082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3e950-11a8-4b45-b103-5da309f2df6e None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:15.737586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3e950-11a8-4b45-b103-5da309f2df6e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:15.737672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3e950-11a8-4b45-b103-5da309f2df6e None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:15.738943 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 553/1111] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:15.743983 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f873037-0fdd-41dd-966e-e476fd91d1a7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:15.744390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f873037-0fdd-41dd-966e-e476fd91d1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:15.744543 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f873037-0fdd-41dd-966e-e476fd91d1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:15.762341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0f873037-0fdd-41dd-966e-e476fd91d1a7 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:15.763538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0f873037-0fdd-41dd-966e-e476fd91d1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:15.829473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 559/1112] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:16.098239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3e950-11a8-4b45-b103-5da309f2df6e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:16.107287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:16.197867 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3e950-11a8-4b45-b103-5da309f2df6e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:16.229764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553e78f-e001-4513-89c0-b540d7e07dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:16.230374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553e78f-e001-4513-89c0-b540d7e07dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:16.230648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553e78f-e001-4513-89c0-b540d7e07dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:16.230916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553e78f-e001-4513-89c0-b540d7e07dd0 None admin] PATH_INFO: `/v3/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:16.232262 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553e78f-e001-4513-89c0-b540d7e07dd0 None admin] RBAC: Authorizing `identity:get_role(role_id=f7bd4a6c906a4ad3b6d2f003f9c37bc5)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:16.233643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553e78f-e001-4513-89c0-b540d7e07dd0 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:16.234931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 554/1113] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:16 2022] GET /identity/v3/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:16.241219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f873037-0fdd-41dd-966e-e476fd91d1a7 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:16.324811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:16.341374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f873037-0fdd-41dd-966e-e476fd91d1a7 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:16.374246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a554988-3631-4337-8053-10f12cf7c41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:16.374803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a554988-3631-4337-8053-10f12cf7c41b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:16.375076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a554988-3631-4337-8053-10f12cf7c41b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:16.375345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a554988-3631-4337-8053-10f12cf7c41b None admin] PATH_INFO: `/v3/users/d53a8fbd99ef4ddd915323b846213e7a` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:16.388243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a554988-3631-4337-8053-10f12cf7c41b None admin] RBAC: Authorizing `identity:get_user(user_id=d53a8fbd99ef4ddd915323b846213e7a)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:16.389703 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a554988-3631-4337-8053-10f12cf7c41b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:16.391274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 560/1114] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:16 2022] GET /identity/v3/users/d53a8fbd99ef4ddd915323b846213e7a => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:16.397131 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553e78f-e001-4513-89c0-b540d7e07dd0 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:16.468522 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:16.493333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553e78f-e001-4513-89c0-b540d7e07dd0 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:16.526345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54554b-05c8-457d-963f-7a88c63eb6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:16.526938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54554b-05c8-457d-963f-7a88c63eb6fc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:16.527216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54554b-05c8-457d-963f-7a88c63eb6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:16.527490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54554b-05c8-457d-963f-7a88c63eb6fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:16.592393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54554b-05c8-457d-963f-7a88c63eb6fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:16.593855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54554b-05c8-457d-963f-7a88c63eb6fc None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:16.666040 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:16.691166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 555/1115] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:20:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:20:16.697102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a554988-3631-4337-8053-10f12cf7c41b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:16.792706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a554988-3631-4337-8053-10f12cf7c41b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:16.825482 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3222c85-99ef-4f90-b68c-44767a4609e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:16.826126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3222c85-99ef-4f90-b68c-44767a4609e4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:16.826406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3222c85-99ef-4f90-b68c-44767a4609e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:16.826679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3222c85-99ef-4f90-b68c-44767a4609e4 None admin] PATH_INFO: `/v3/projects/158c81a6e0a84b05b14281c3abafce78` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:16.827777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3222c85-99ef-4f90-b68c-44767a4609e4 None admin] RBAC: Authorizing `identity:get_project(project_id=158c81a6e0a84b05b14281c3abafce78)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:16.829238 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3222c85-99ef-4f90-b68c-44767a4609e4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:16.831056 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 561/1116] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:20:16 2022] GET /identity/v3/projects/158c81a6e0a84b05b14281c3abafce78 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:16.837198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54554b-05c8-457d-963f-7a88c63eb6fc None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:16.932397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54554b-05c8-457d-963f-7a88c63eb6fc None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:16.965086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7c11f-4150-4df8-84ea-8b478880c4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:16.965682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7c11f-4150-4df8-84ea-8b478880c4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:16.965960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7c11f-4150-4df8-84ea-8b478880c4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:16.966231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7c11f-4150-4df8-84ea-8b478880c4d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:16.966944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7c11f-4150-4df8-84ea-8b478880c4d1 None admin] RBAC: Adding query filter params (role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5, scope.project.id=158c81a6e0a84b05b14281c3abafce78, user.id=d53a8fbd99ef4ddd915323b846213e7a) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:16.967252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7c11f-4150-4df8-84ea-8b478880c4d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:16.968669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7c11f-4150-4df8-84ea-8b478880c4d1 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:16.974587 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 556/1117] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:20:16 2022] GET /identity/v3/role_assignments?user.id=d53a8fbd99ef4ddd915323b846213e7a&scope.project.id=158c81a6e0a84b05b14281c3abafce78&role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:18.816836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3950f-65ad-4d96-8892-4950fac1cdb9 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:18.817394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3950f-65ad-4d96-8892-4950fac1cdb9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:18.817394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3950f-65ad-4d96-8892-4950fac1cdb9 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:18.818736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 562/1118] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:18.823785 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeba386-e0e4-483b-aedf-eef893b199f1 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:18.824204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeba386-e0e4-483b-aedf-eef893b199f1 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:18.824336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeba386-e0e4-483b-aedf-eef893b199f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:18.842768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3aeba386-e0e4-483b-aedf-eef893b199f1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:18.843945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3aeba386-e0e4-483b-aedf-eef893b199f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:18.872611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:18.909348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 557/1119] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:19.190998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3950f-65ad-4d96-8892-4950fac1cdb9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:19.205502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:19.294346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3950f-65ad-4d96-8892-4950fac1cdb9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:19.328056 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812cdc3-9775-4d6b-908b-131ddcf9541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:19.328677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812cdc3-9775-4d6b-908b-131ddcf9541e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:19.328987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812cdc3-9775-4d6b-908b-131ddcf9541e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:19.329263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812cdc3-9775-4d6b-908b-131ddcf9541e None admin] PATH_INFO: `/v3/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:19.331139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812cdc3-9775-4d6b-908b-131ddcf9541e None admin] RBAC: Authorizing `identity:get_role(role_id=f7bd4a6c906a4ad3b6d2f003f9c37bc5)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:19.332499 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812cdc3-9775-4d6b-908b-131ddcf9541e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:19.333829 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 563/1120] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:19 2022] GET /identity/v3/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:19.339598 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeba386-e0e4-483b-aedf-eef893b199f1 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:19.434832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeba386-e0e4-483b-aedf-eef893b199f1 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:19.468035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afaf7c0-9908-4c31-a1bf-fed6684f8c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:19.468589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afaf7c0-9908-4c31-a1bf-fed6684f8c33 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:19.468894 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afaf7c0-9908-4c31-a1bf-fed6684f8c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:19.469169 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afaf7c0-9908-4c31-a1bf-fed6684f8c33 None admin] PATH_INFO: `/v3/users/d53a8fbd99ef4ddd915323b846213e7a` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:19.470275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afaf7c0-9908-4c31-a1bf-fed6684f8c33 None admin] RBAC: Authorizing `identity:get_user(user_id=d53a8fbd99ef4ddd915323b846213e7a)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:19.472261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afaf7c0-9908-4c31-a1bf-fed6684f8c33 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:19.474023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 558/1121] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:19 2022] GET /identity/v3/users/d53a8fbd99ef4ddd915323b846213e7a => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:19.479941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812cdc3-9775-4d6b-908b-131ddcf9541e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:19.556233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:19.577525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812cdc3-9775-4d6b-908b-131ddcf9541e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:19.610372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614e0ee-f10d-4640-ab6f-707b6de65690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:19.610993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614e0ee-f10d-4640-ab6f-707b6de65690 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:19.611892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614e0ee-f10d-4640-ab6f-707b6de65690 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:19.612162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614e0ee-f10d-4640-ab6f-707b6de65690 None admin] PATH_INFO: `/v3/projects/158c81a6e0a84b05b14281c3abafce78` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:19.613284 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614e0ee-f10d-4640-ab6f-707b6de65690 None admin] RBAC: Authorizing `identity:get_project(project_id=158c81a6e0a84b05b14281c3abafce78)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:19.614705 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614e0ee-f10d-4640-ab6f-707b6de65690 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:19.616295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 564/1122] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:20:19 2022] GET /identity/v3/projects/158c81a6e0a84b05b14281c3abafce78 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:19.622217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afaf7c0-9908-4c31-a1bf-fed6684f8c33 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:19.656094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:19.719213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afaf7c0-9908-4c31-a1bf-fed6684f8c33 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:19.753893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625fa78-60da-4e25-b64a-05f50e5f91c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:19.754538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625fa78-60da-4e25-b64a-05f50e5f91c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:19.754818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625fa78-60da-4e25-b64a-05f50e5f91c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:19.755089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625fa78-60da-4e25-b64a-05f50e5f91c6 None admin] PATH_INFO: `/v3/projects/158c81a6e0a84b05b14281c3abafce78/users/d53a8fbd99ef4ddd915323b846213e7a/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:19.757220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4625fa78-60da-4e25-b64a-05f50e5f91c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=158c81a6e0a84b05b14281c3abafce78, user_id=d53a8fbd99ef4ddd915323b846213e7a, role_id=f7bd4a6c906a4ad3b6d2f003f9c37bc5)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:19.758646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4625fa78-60da-4e25-b64a-05f50e5f91c6 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:19.770652 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 559/1123] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:20:19 2022] PUT /identity/v3/projects/158c81a6e0a84b05b14281c3abafce78/users/d53a8fbd99ef4ddd915323b846213e7a/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:20:21.641802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbfc39-0982-4473-a569-529994a08035 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:21.642371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbfc39-0982-4473-a569-529994a08035 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:21.642371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbfc39-0982-4473-a569-529994a08035 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:21.643690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 565/1124] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:21.649124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff1cf9-2529-42a9-a014-83b88e29bd3c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:21.649512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff1cf9-2529-42a9-a014-83b88e29bd3c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:21.649678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff1cf9-2529-42a9-a014-83b88e29bd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:21.667527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-28ff1cf9-2529-42a9-a014-83b88e29bd3c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:21.668714 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-28ff1cf9-2529-42a9-a014-83b88e29bd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:21.733000 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 560/1125] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:22.003766 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbfc39-0982-4473-a569-529994a08035 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:22.103175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbfc39-0982-4473-a569-529994a08035 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:22.135983 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2cb27-7fbf-49b3-a7a0-7db7b1b7930a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:22.136596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2cb27-7fbf-49b3-a7a0-7db7b1b7930a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:22.136902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2cb27-7fbf-49b3-a7a0-7db7b1b7930a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:22.137177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2cb27-7fbf-49b3-a7a0-7db7b1b7930a None admin] PATH_INFO: `/v3/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:22.138549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2cb27-7fbf-49b3-a7a0-7db7b1b7930a None admin] RBAC: Authorizing `identity:get_role(role_id=f7bd4a6c906a4ad3b6d2f003f9c37bc5)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:22.139916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2cb27-7fbf-49b3-a7a0-7db7b1b7930a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:22.141227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 566/1126] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:22 2022] GET /identity/v3/roles/f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:22.147034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff1cf9-2529-42a9-a014-83b88e29bd3c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:22.173152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:22.243983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff1cf9-2529-42a9-a014-83b88e29bd3c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:22.276860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86910841-2c9e-41d5-981b-4a0b9a1ca708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:22.277414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86910841-2c9e-41d5-981b-4a0b9a1ca708 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:22.277688 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86910841-2c9e-41d5-981b-4a0b9a1ca708 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:22.277954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86910841-2c9e-41d5-981b-4a0b9a1ca708 None admin] PATH_INFO: `/v3/users/d53a8fbd99ef4ddd915323b846213e7a` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:22.279044 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86910841-2c9e-41d5-981b-4a0b9a1ca708 None admin] RBAC: Authorizing `identity:get_user(user_id=d53a8fbd99ef4ddd915323b846213e7a)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:22.280453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86910841-2c9e-41d5-981b-4a0b9a1ca708 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:22.282031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 561/1127] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:22 2022] GET /identity/v3/users/d53a8fbd99ef4ddd915323b846213e7a => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:22.288394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2cb27-7fbf-49b3-a7a0-7db7b1b7930a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:22.303957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:22.385149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2cb27-7fbf-49b3-a7a0-7db7b1b7930a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:22.419389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b827e-7c81-4c5c-a728-076ce39f6674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:22.419978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b827e-7c81-4c5c-a728-076ce39f6674 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:22.420253 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b827e-7c81-4c5c-a728-076ce39f6674 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:22.420522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b827e-7c81-4c5c-a728-076ce39f6674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:22.485067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b827e-7c81-4c5c-a728-076ce39f6674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:22.486678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b827e-7c81-4c5c-a728-076ce39f6674 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:22.500159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:22.586184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 567/1128] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:20:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:20:22.592101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86910841-2c9e-41d5-981b-4a0b9a1ca708 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:22.682763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:22.690323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86910841-2c9e-41d5-981b-4a0b9a1ca708 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:22.722904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ba5e8-88bf-4105-8b85-c98fd5254f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:22.723510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ba5e8-88bf-4105-8b85-c98fd5254f2f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:22.723786 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ba5e8-88bf-4105-8b85-c98fd5254f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:22.724054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ba5e8-88bf-4105-8b85-c98fd5254f2f None admin] PATH_INFO: `/v3/projects/158c81a6e0a84b05b14281c3abafce78` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:22.725178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ba5e8-88bf-4105-8b85-c98fd5254f2f None admin] RBAC: Authorizing `identity:get_project(project_id=158c81a6e0a84b05b14281c3abafce78)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:22.726876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ba5e8-88bf-4105-8b85-c98fd5254f2f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:22.728474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 562/1129] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:20:22 2022] GET /identity/v3/projects/158c81a6e0a84b05b14281c3abafce78 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:22.734651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b827e-7c81-4c5c-a728-076ce39f6674 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:22.830611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b827e-7c81-4c5c-a728-076ce39f6674 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:22.843991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:22.863814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c5b01-451b-41ad-bdcf-44a672dcf7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:22.864413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c5b01-451b-41ad-bdcf-44a672dcf7ee None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:22.864717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c5b01-451b-41ad-bdcf-44a672dcf7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:22.865018 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c5b01-451b-41ad-bdcf-44a672dcf7ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:22.865736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c5b01-451b-41ad-bdcf-44a672dcf7ee None admin] RBAC: Adding query filter params (role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5, scope.project.id=158c81a6e0a84b05b14281c3abafce78, user.id=d53a8fbd99ef4ddd915323b846213e7a) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:22.866042 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c5b01-451b-41ad-bdcf-44a672dcf7ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:22.867464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c5b01-451b-41ad-bdcf-44a672dcf7ee None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:22.873054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 568/1130] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:20:22 2022] GET /identity/v3/role_assignments?user.id=d53a8fbd99ef4ddd915323b846213e7a&scope.project.id=158c81a6e0a84b05b14281c3abafce78&role.id=f7bd4a6c906a4ad3b6d2f003f9c37bc5 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:24.748927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff78d3b-467d-4676-bdb1-12a13bb9ecdd None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:24.749426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff78d3b-467d-4676-bdb1-12a13bb9ecdd None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:24.749517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff78d3b-467d-4676-bdb1-12a13bb9ecdd None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:24.750796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 563/1131] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:24.756729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9cb5f-5ce9-4fa2-8054-a7319318be38 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:24.757151 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9cb5f-5ce9-4fa2-8054-a7319318be38 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:24.757306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9cb5f-5ce9-4fa2-8054-a7319318be38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:24.774916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-37b9cb5f-5ce9-4fa2-8054-a7319318be38 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:24.776105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-37b9cb5f-5ce9-4fa2-8054-a7319318be38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:24.840143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 569/1132] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:25.112080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff78d3b-467d-4676-bdb1-12a13bb9ecdd None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:25.210082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff78d3b-467d-4676-bdb1-12a13bb9ecdd None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:25.242789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a62e1-5744-489a-8d30-c4de9d625ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:25.243403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a62e1-5744-489a-8d30-c4de9d625ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:25.243680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a62e1-5744-489a-8d30-c4de9d625ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:25.243955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a62e1-5744-489a-8d30-c4de9d625ae6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:25.245096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a62e1-5744-489a-8d30-c4de9d625ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:25.246523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a62e1-5744-489a-8d30-c4de9d625ae6 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:25.248057 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 564/1133] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:25.254207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9cb5f-5ce9-4fa2-8054-a7319318be38 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:25.352215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9cb5f-5ce9-4fa2-8054-a7319318be38 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:25.361501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:25.387495 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b3e2a-dff1-4842-bfd9-cc4b087442ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:25.388037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b3e2a-dff1-4842-bfd9-cc4b087442ba None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:25.388317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b3e2a-dff1-4842-bfd9-cc4b087442ba None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:25.388587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b3e2a-dff1-4842-bfd9-cc4b087442ba None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:25.389257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b3e2a-dff1-4842-bfd9-cc4b087442ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:25.390689 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b3e2a-dff1-4842-bfd9-cc4b087442ba None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:25.418309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 570/1134] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:20:25 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:20:27.326949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a4f5b-1c9d-4419-97ac-6a3b5e75af6a None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:27.327471 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a4f5b-1c9d-4419-97ac-6a3b5e75af6a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:27.327471 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a4f5b-1c9d-4419-97ac-6a3b5e75af6a None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:27.328816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 565/1135] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:27.334613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb75ee-bedf-4f38-8fb0-085f3e2a53e9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:27.335037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb75ee-bedf-4f38-8fb0-085f3e2a53e9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:27.335173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb75ee-bedf-4f38-8fb0-085f3e2a53e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:27.353182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-16fb75ee-bedf-4f38-8fb0-085f3e2a53e9 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:27.354375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-16fb75ee-bedf-4f38-8fb0-085f3e2a53e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:27.419146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 571/1136] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:27.690518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a4f5b-1c9d-4419-97ac-6a3b5e75af6a None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:27.719915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:27.791732 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a4f5b-1c9d-4419-97ac-6a3b5e75af6a None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:27.825638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ae6f4-75e5-49c1-b287-1356e32a6ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:27.826258 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ae6f4-75e5-49c1-b287-1356e32a6ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:27.826544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ae6f4-75e5-49c1-b287-1356e32a6ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:27.826818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ae6f4-75e5-49c1-b287-1356e32a6ba4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:27.827926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ae6f4-75e5-49c1-b287-1356e32a6ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:27.829417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ae6f4-75e5-49c1-b287-1356e32a6ba4 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:27.831070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 566/1137] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:20:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:27.836863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb75ee-bedf-4f38-8fb0-085f3e2a53e9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:27.925010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:27.935594 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb75ee-bedf-4f38-8fb0-085f3e2a53e9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:27.967647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb68fce-f16f-4349-aa4d-14d9b5065dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:27.968239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb68fce-f16f-4349-aa4d-14d9b5065dc5 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:27.968513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb68fce-f16f-4349-aa4d-14d9b5065dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:27.968806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb68fce-f16f-4349-aa4d-14d9b5065dc5 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:27.969440 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb68fce-f16f-4349-aa4d-14d9b5065dc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:27.971350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb68fce-f16f-4349-aa4d-14d9b5065dc5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:27.992340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 572/1138] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:20:27 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:20:29.909957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf9ad5-a497-4e08-8fe5-20fb27bac26e None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:29.910449 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf9ad5-a497-4e08-8fe5-20fb27bac26e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:29.910449 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf9ad5-a497-4e08-8fe5-20fb27bac26e None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:29.911750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 567/1139] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:29.916830 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e2a2e-6e8a-4f9d-a3e1-50b02732cbfb None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:29.917215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e2a2e-6e8a-4f9d-a3e1-50b02732cbfb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:29.917382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e2a2e-6e8a-4f9d-a3e1-50b02732cbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:29.935020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a83e2a2e-6e8a-4f9d-a3e1-50b02732cbfb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:29.936212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a83e2a2e-6e8a-4f9d-a3e1-50b02732cbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:30.001303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 573/1140] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:30.266121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf9ad5-a497-4e08-8fe5-20fb27bac26e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:30.360031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:30.365007 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf9ad5-a497-4e08-8fe5-20fb27bac26e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:30.398306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e9052-bfd1-4bfd-95fe-13886496ff38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:30.398917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e9052-bfd1-4bfd-95fe-13886496ff38 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:30.399195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e9052-bfd1-4bfd-95fe-13886496ff38 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:30.399464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e9052-bfd1-4bfd-95fe-13886496ff38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:30.409377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e9052-bfd1-4bfd-95fe-13886496ff38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-4b4e9052-bfd1-4bfd-95fe-13886496ff38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:20:30.414027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:30.415196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:30.416467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 568/1141] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:20:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:30.421009 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e2a2e-6e8a-4f9d-a3e1-50b02732cbfb None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:30.494773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:30.518357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e2a2e-6e8a-4f9d-a3e1-50b02732cbfb None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:30.550994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144ffbc-e067-4f59-9185-ffa42b72d840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:30.551584 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144ffbc-e067-4f59-9185-ffa42b72d840 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:30.551862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144ffbc-e067-4f59-9185-ffa42b72d840 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:30.552134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144ffbc-e067-4f59-9185-ffa42b72d840 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:30.552806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144ffbc-e067-4f59-9185-ffa42b72d840 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:30.553106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144ffbc-e067-4f59-9185-ffa42b72d840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:30.554455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144ffbc-e067-4f59-9185-ffa42b72d840 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:30.562157 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 574/1142] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:20:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:30.568452 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e9052-bfd1-4bfd-95fe-13886496ff38 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:30.670390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e9052-bfd1-4bfd-95fe-13886496ff38 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:30.704581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af7da9-7f87-4de4-a3d2-68cd6f821dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:30.705359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af7da9-7f87-4de4-a3d2-68cd6f821dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:30.705639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af7da9-7f87-4de4-a3d2-68cd6f821dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:30.705912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af7da9-7f87-4de4-a3d2-68cd6f821dd5 None admin] PATH_INFO: `/v3/users/8c6e295ddc99496d9373aa632e45f4cf` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:30.719080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af7da9-7f87-4de4-a3d2-68cd6f821dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=8c6e295ddc99496d9373aa632e45f4cf)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:30.721211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af7da9-7f87-4de4-a3d2-68cd6f821dd5 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:30.722796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 569/1143] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:30 2022] GET /identity/v3/users/8c6e295ddc99496d9373aa632e45f4cf => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:30.728977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144ffbc-e067-4f59-9185-ffa42b72d840 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:30.826876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144ffbc-e067-4f59-9185-ffa42b72d840 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:30.859346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccc547-59b7-4feb-b7a3-620ae28422db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:30.859956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccc547-59b7-4feb-b7a3-620ae28422db None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:30.860240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccc547-59b7-4feb-b7a3-620ae28422db None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:30.861132 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccc547-59b7-4feb-b7a3-620ae28422db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:30.869298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:30.926019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccc547-59b7-4feb-b7a3-620ae28422db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:30.927448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccc547-59b7-4feb-b7a3-620ae28422db None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:31.027957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 575/1144] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:20:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:20:31.034432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af7da9-7f87-4de4-a3d2-68cd6f821dd5 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:31.052363 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:31.131297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af7da9-7f87-4de4-a3d2-68cd6f821dd5 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:31.164278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e9fda-8a06-45ac-b706-526d2b7b1b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:31.164998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e9fda-8a06-45ac-b706-526d2b7b1b32 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:31.165276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e9fda-8a06-45ac-b706-526d2b7b1b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:31.165544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e9fda-8a06-45ac-b706-526d2b7b1b32 None admin] PATH_INFO: `/v3/projects/21616bd2cf084aaab64789e8361aef03` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:31.166649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e9fda-8a06-45ac-b706-526d2b7b1b32 None admin] RBAC: Authorizing `identity:get_project(project_id=21616bd2cf084aaab64789e8361aef03)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:31.168055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e9fda-8a06-45ac-b706-526d2b7b1b32 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:31.169690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 570/1145] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:20:31 2022] GET /identity/v3/projects/21616bd2cf084aaab64789e8361aef03 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:31.176437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccc547-59b7-4feb-b7a3-620ae28422db None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:31.215007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:31.275234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccc547-59b7-4feb-b7a3-620ae28422db None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:31.309911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2724ae9-a041-476a-899c-c60a9d895a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:31.310508 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2724ae9-a041-476a-899c-c60a9d895a43 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:31.310792 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2724ae9-a041-476a-899c-c60a9d895a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:31.311070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2724ae9-a041-476a-899c-c60a9d895a43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:31.311787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2724ae9-a041-476a-899c-c60a9d895a43 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=21616bd2cf084aaab64789e8361aef03, user.id=8c6e295ddc99496d9373aa632e45f4cf) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:31.312097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2724ae9-a041-476a-899c-c60a9d895a43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:31.313591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2724ae9-a041-476a-899c-c60a9d895a43 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:31.319098 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 576/1146] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:20:31 2022] GET /identity/v3/role_assignments?user.id=8c6e295ddc99496d9373aa632e45f4cf&scope.project.id=21616bd2cf084aaab64789e8361aef03&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:33.155480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f15a8a-cbca-40d0-9627-1703deb749d5 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:33.156050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f15a8a-cbca-40d0-9627-1703deb749d5 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:33.156050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f15a8a-cbca-40d0-9627-1703deb749d5 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:33.157360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 571/1147] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:33.162519 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3e8a6-0fb6-4214-a65e-73cdcc7b8894 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:33.162896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3e8a6-0fb6-4214-a65e-73cdcc7b8894 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:33.163066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3e8a6-0fb6-4214-a65e-73cdcc7b8894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:33.189393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f8a3e8a6-0fb6-4214-a65e-73cdcc7b8894 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:33.190585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f8a3e8a6-0fb6-4214-a65e-73cdcc7b8894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:33.255528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 577/1148] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:33.519726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f15a8a-cbca-40d0-9627-1703deb749d5 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:33.602216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:33.618987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f15a8a-cbca-40d0-9627-1703deb749d5 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:33.654092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac1ddc-cd9c-480a-b55c-28de13ea07fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:33.654712 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac1ddc-cd9c-480a-b55c-28de13ea07fb None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:33.654987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac1ddc-cd9c-480a-b55c-28de13ea07fb None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:33.655256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac1ddc-cd9c-480a-b55c-28de13ea07fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:33.665325 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac1ddc-cd9c-480a-b55c-28de13ea07fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-76ac1ddc-cd9c-480a-b55c-28de13ea07fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:20:33.669941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:33.671025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:33.672205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 572/1149] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:20:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:33.677511 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3e8a6-0fb6-4214-a65e-73cdcc7b8894 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:33.707076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:33.772941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3e8a6-0fb6-4214-a65e-73cdcc7b8894 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:33.805403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666200c6-9963-4513-87fd-db26848e6abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:33.805998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666200c6-9963-4513-87fd-db26848e6abd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:33.806279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666200c6-9963-4513-87fd-db26848e6abd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:33.806545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666200c6-9963-4513-87fd-db26848e6abd None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:33.807193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666200c6-9963-4513-87fd-db26848e6abd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:33.807489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666200c6-9963-4513-87fd-db26848e6abd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:33.808872 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666200c6-9963-4513-87fd-db26848e6abd None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:33.817252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 578/1150] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:20:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:33.822976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac1ddc-cd9c-480a-b55c-28de13ea07fb None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:33.919611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac1ddc-cd9c-480a-b55c-28de13ea07fb None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:33.952252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f46b33-2722-4cc9-9657-96e9aabd7488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:33.952889 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f46b33-2722-4cc9-9657-96e9aabd7488 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:33.953170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f46b33-2722-4cc9-9657-96e9aabd7488 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:33.953442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f46b33-2722-4cc9-9657-96e9aabd7488 None admin] PATH_INFO: `/v3/users/8c6e295ddc99496d9373aa632e45f4cf` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:33.954560 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f46b33-2722-4cc9-9657-96e9aabd7488 None admin] RBAC: Authorizing `identity:get_user(user_id=8c6e295ddc99496d9373aa632e45f4cf)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:33.955972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f46b33-2722-4cc9-9657-96e9aabd7488 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:33.957536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 573/1151] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:33 2022] GET /identity/v3/users/8c6e295ddc99496d9373aa632e45f4cf => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:33.963336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666200c6-9963-4513-87fd-db26848e6abd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:34.053365 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:34.060054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666200c6-9963-4513-87fd-db26848e6abd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:34.091761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096aed3-5d24-41e3-b6c1-640707f3137d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:34.092371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096aed3-5d24-41e3-b6c1-640707f3137d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:34.092648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096aed3-5d24-41e3-b6c1-640707f3137d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:34.092964 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096aed3-5d24-41e3-b6c1-640707f3137d None admin] PATH_INFO: `/v3/projects/21616bd2cf084aaab64789e8361aef03` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:34.094058 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096aed3-5d24-41e3-b6c1-640707f3137d None admin] RBAC: Authorizing `identity:get_project(project_id=21616bd2cf084aaab64789e8361aef03)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:34.096004 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096aed3-5d24-41e3-b6c1-640707f3137d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:34.097674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 579/1152] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:20:33 2022] GET /identity/v3/projects/21616bd2cf084aaab64789e8361aef03 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:34.103686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f46b33-2722-4cc9-9657-96e9aabd7488 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:34.118438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:34.198463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f46b33-2722-4cc9-9657-96e9aabd7488 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:34.231918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4f015-19a5-4c69-ad3c-7278400734ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:34.232549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4f015-19a5-4c69-ad3c-7278400734ea None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:34.232855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4f015-19a5-4c69-ad3c-7278400734ea None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:34.233123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4f015-19a5-4c69-ad3c-7278400734ea None admin] PATH_INFO: `/v3/projects/21616bd2cf084aaab64789e8361aef03/users/8c6e295ddc99496d9373aa632e45f4cf/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:34.234977 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4f015-19a5-4c69-ad3c-7278400734ea None admin] RBAC: Authorizing `identity:create_grant(project_id=21616bd2cf084aaab64789e8361aef03, user_id=8c6e295ddc99496d9373aa632e45f4cf, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:34.236670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4f015-19a5-4c69-ad3c-7278400734ea None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:34.248761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 574/1153] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:20:34 2022] PUT /identity/v3/projects/21616bd2cf084aaab64789e8361aef03/users/8c6e295ddc99496d9373aa632e45f4cf/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:20:36.116905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cf99b-c509-4146-97b8-cfcb7d9290e4 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:36.117446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cf99b-c509-4146-97b8-cfcb7d9290e4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:36.117446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cf99b-c509-4146-97b8-cfcb7d9290e4 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:36.118798 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 580/1154] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:36.124063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7aa485-d1e4-41ca-aefb-a4784a455a3c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:36.124482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7aa485-d1e4-41ca-aefb-a4784a455a3c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:36.124632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7aa485-d1e4-41ca-aefb-a4784a455a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:36.142215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-0b7aa485-d1e4-41ca-aefb-a4784a455a3c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:36.143385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0b7aa485-d1e4-41ca-aefb-a4784a455a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:36.206710 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 575/1155] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:36.471928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cf99b-c509-4146-97b8-cfcb7d9290e4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:36.571406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cf99b-c509-4146-97b8-cfcb7d9290e4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:36.604271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d268d-1dc6-40fa-82f1-1721d3655acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:36.604917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d268d-1dc6-40fa-82f1-1721d3655acd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:36.605197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d268d-1dc6-40fa-82f1-1721d3655acd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:36.605470 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d268d-1dc6-40fa-82f1-1721d3655acd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:36.615728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d268d-1dc6-40fa-82f1-1721d3655acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-047d268d-1dc6-40fa-82f1-1721d3655acd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:20:36.620352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:36.621446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:36.622466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 581/1156] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:20:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:36.627692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7aa485-d1e4-41ca-aefb-a4784a455a3c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:36.629893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:36.726796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7aa485-d1e4-41ca-aefb-a4784a455a3c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:36.758969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0db1f-79c9-4fc4-91fc-08fba6b6c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:36.759568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0db1f-79c9-4fc4-91fc-08fba6b6c541 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:36.759848 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0db1f-79c9-4fc4-91fc-08fba6b6c541 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:36.760121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0db1f-79c9-4fc4-91fc-08fba6b6c541 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:36.760799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0db1f-79c9-4fc4-91fc-08fba6b6c541 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:36.761102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0db1f-79c9-4fc4-91fc-08fba6b6c541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:36.762461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0db1f-79c9-4fc4-91fc-08fba6b6c541 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:36.770836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 576/1157] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:20:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:36.776574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d268d-1dc6-40fa-82f1-1721d3655acd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:36.801455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:36.874552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d268d-1dc6-40fa-82f1-1721d3655acd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:36.907853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaffe8f-26eb-4f60-9bf2-3008d13bf598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:36.908468 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaffe8f-26eb-4f60-9bf2-3008d13bf598 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:36.908775 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaffe8f-26eb-4f60-9bf2-3008d13bf598 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:36.909054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaffe8f-26eb-4f60-9bf2-3008d13bf598 None admin] PATH_INFO: `/v3/users/8c6e295ddc99496d9373aa632e45f4cf` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:36.910165 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaffe8f-26eb-4f60-9bf2-3008d13bf598 None admin] RBAC: Authorizing `identity:get_user(user_id=8c6e295ddc99496d9373aa632e45f4cf)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:36.911594 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaffe8f-26eb-4f60-9bf2-3008d13bf598 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:36.913183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 582/1158] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:36 2022] GET /identity/v3/users/8c6e295ddc99496d9373aa632e45f4cf => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:36.918944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0db1f-79c9-4fc4-91fc-08fba6b6c541 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:36.979847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:37.017180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0db1f-79c9-4fc4-91fc-08fba6b6c541 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:37.048736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c7520-edba-4b67-825e-2380e58d95cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:37.049355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c7520-edba-4b67-825e-2380e58d95cf None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:37.049633 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c7520-edba-4b67-825e-2380e58d95cf None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:37.049904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c7520-edba-4b67-825e-2380e58d95cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:37.116193 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c7520-edba-4b67-825e-2380e58d95cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:37.117664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c7520-edba-4b67-825e-2380e58d95cf None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:37.176386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:37.214631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 577/1159] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:20:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:20:37.220654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaffe8f-26eb-4f60-9bf2-3008d13bf598 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:37.308271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:37.318905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaffe8f-26eb-4f60-9bf2-3008d13bf598 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:37.350782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bfec7-4e2b-4615-b92e-bc66320fc1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:37.351395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bfec7-4e2b-4615-b92e-bc66320fc1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:37.351672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bfec7-4e2b-4615-b92e-bc66320fc1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:37.351941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bfec7-4e2b-4615-b92e-bc66320fc1d3 None admin] PATH_INFO: `/v3/projects/21616bd2cf084aaab64789e8361aef03` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:37.353054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bfec7-4e2b-4615-b92e-bc66320fc1d3 None admin] RBAC: Authorizing `identity:get_project(project_id=21616bd2cf084aaab64789e8361aef03)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:37.354471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bfec7-4e2b-4615-b92e-bc66320fc1d3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:37.356081 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 583/1160] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:20:37 2022] GET /identity/v3/projects/21616bd2cf084aaab64789e8361aef03 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:37.362694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c7520-edba-4b67-825e-2380e58d95cf None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:37.459436 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c7520-edba-4b67-825e-2380e58d95cf None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:37.492317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cc491-cd2e-4d69-ad98-a425b9399b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:37.492930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cc491-cd2e-4d69-ad98-a425b9399b03 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:37.493211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cc491-cd2e-4d69-ad98-a425b9399b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:37.493482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cc491-cd2e-4d69-ad98-a425b9399b03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:37.494187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cc491-cd2e-4d69-ad98-a425b9399b03 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=21616bd2cf084aaab64789e8361aef03, user.id=8c6e295ddc99496d9373aa632e45f4cf) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:37.494488 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cc491-cd2e-4d69-ad98-a425b9399b03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:37.495897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cc491-cd2e-4d69-ad98-a425b9399b03 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:37.502187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 578/1161] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:20:37 2022] GET /identity/v3/role_assignments?user.id=8c6e295ddc99496d9373aa632e45f4cf&scope.project.id=21616bd2cf084aaab64789e8361aef03&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:39.379706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3227e1-b337-4fb2-8b96-cc95a6da1b1c None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:39.380259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3227e1-b337-4fb2-8b96-cc95a6da1b1c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:39.380259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3227e1-b337-4fb2-8b96-cc95a6da1b1c None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:39.381727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 584/1162] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:39.386742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac7e21-f8b2-44f0-8a43-16ab5e038de8 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:39.387162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac7e21-f8b2-44f0-8a43-16ab5e038de8 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:39.387270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac7e21-f8b2-44f0-8a43-16ab5e038de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:39.404988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-67ac7e21-f8b2-44f0-8a43-16ab5e038de8 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:39.406167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-67ac7e21-f8b2-44f0-8a43-16ab5e038de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:39.429117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:39.472980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 579/1163] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:39.743189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3227e1-b337-4fb2-8b96-cc95a6da1b1c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:39.843087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3227e1-b337-4fb2-8b96-cc95a6da1b1c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:39.875929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1953b-2803-4921-b3df-be65428377ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:39.876530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1953b-2803-4921-b3df-be65428377ee None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:39.876834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1953b-2803-4921-b3df-be65428377ee None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:39.877111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1953b-2803-4921-b3df-be65428377ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:39.903222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:39.944234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1953b-2803-4921-b3df-be65428377ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:39.945727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1953b-2803-4921-b3df-be65428377ee None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:40.045814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 585/1164] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:20:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:20:40.052023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac7e21-f8b2-44f0-8a43-16ab5e038de8 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:40.148211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac7e21-f8b2-44f0-8a43-16ab5e038de8 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:40.181721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897820d3-5cbd-4b58-84f4-12952b0c85a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:40.182277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897820d3-5cbd-4b58-84f4-12952b0c85a9 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:40.182554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897820d3-5cbd-4b58-84f4-12952b0c85a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:40.182824 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897820d3-5cbd-4b58-84f4-12952b0c85a9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:40.190049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897820d3-5cbd-4b58-84f4-12952b0c85a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:40.191476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897820d3-5cbd-4b58-84f4-12952b0c85a9 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-897820d3-5cbd-4b58-84f4-12952b0c85a9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:40.202481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:40.203756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:40.204923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 580/1165] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Aug 19 16:20:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:40.209410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1953b-2803-4921-b3df-be65428377ee None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:40.265459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:40.305906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1953b-2803-4921-b3df-be65428377ee None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:40.338464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ead309-9a01-49a6-bd0a-430a8e535583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:40.339056 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ead309-9a01-49a6-bd0a-430a8e535583 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:40.339332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ead309-9a01-49a6-bd0a-430a8e535583 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:40.339602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ead309-9a01-49a6-bd0a-430a8e535583 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:40.340228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ead309-9a01-49a6-bd0a-430a8e535583 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:40.340524 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ead309-9a01-49a6-bd0a-430a8e535583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:40.341912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ead309-9a01-49a6-bd0a-430a8e535583 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:40.348839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 586/1166] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Aug 19 16:20:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:42.213592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1593506-28b1-4b2d-a0d3-d565fa495f8c None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:42.214120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1593506-28b1-4b2d-a0d3-d565fa495f8c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:42.214120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1593506-28b1-4b2d-a0d3-d565fa495f8c None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:42.215481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 581/1167] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:42.221057 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef066e54-d55a-4a26-8b2a-e07a0c21dac8 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:42.221484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef066e54-d55a-4a26-8b2a-e07a0c21dac8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:42.221614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef066e54-d55a-4a26-8b2a-e07a0c21dac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:42.240686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ef066e54-d55a-4a26-8b2a-e07a0c21dac8 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:42.242375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ef066e54-d55a-4a26-8b2a-e07a0c21dac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:42.310957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 587/1168] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:42.579616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1593506-28b1-4b2d-a0d3-d565fa495f8c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:42.598967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:42.679841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1593506-28b1-4b2d-a0d3-d565fa495f8c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:42.712481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3155f-d658-4795-99c4-4a61172c59e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:42.713105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3155f-d658-4795-99c4-4a61172c59e4 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:42.713383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3155f-d658-4795-99c4-4a61172c59e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:42.713654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3155f-d658-4795-99c4-4a61172c59e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:42.714313 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3155f-d658-4795-99c4-4a61172c59e4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:42.715672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3155f-d658-4795-99c4-4a61172c59e4 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:42.731755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 582/1169] 10.222.0.55 () {60 vars in 1138 bytes} [Fri Aug 19 16:20:42 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:20:44.644643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d7425-8540-41d4-999d-ae23c7082c90 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:44.645183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d7425-8540-41d4-999d-ae23c7082c90 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:44.645270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d7425-8540-41d4-999d-ae23c7082c90 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:44.646560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 588/1170] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:44.651725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5eae6-e1ac-444b-87a9-dcb8dffa2127 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:44.652142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5eae6-e1ac-444b-87a9-dcb8dffa2127 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:44.652277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5eae6-e1ac-444b-87a9-dcb8dffa2127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:44.669778 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d0a5eae6-e1ac-444b-87a9-dcb8dffa2127 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:44.670947 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d0a5eae6-e1ac-444b-87a9-dcb8dffa2127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:44.735646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 583/1171] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:45.005590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d7425-8540-41d4-999d-ae23c7082c90 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:45.057624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:45.114453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d7425-8540-41d4-999d-ae23c7082c90 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:45.148585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79566e0e-9bcb-4814-99c5-669ae582aaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:45.149377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79566e0e-9bcb-4814-99c5-669ae582aaee None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:45.149653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79566e0e-9bcb-4814-99c5-669ae582aaee None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:45.149924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79566e0e-9bcb-4814-99c5-669ae582aaee None admin] PATH_INFO: `/v3/domains/83dff104af5747c495b49785090514d5` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:45.159904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79566e0e-9bcb-4814-99c5-669ae582aaee None admin] RBAC: Authorizing `identity:get_domain(domain_id=83dff104af5747c495b49785090514d5)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:45.161372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79566e0e-9bcb-4814-99c5-669ae582aaee None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:45.162943 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 589/1172] 10.222.0.55 () {56 vars in 1183 bytes} [Fri Aug 19 16:20:45 2022] GET /identity/v3/domains/83dff104af5747c495b49785090514d5 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:45.168791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5eae6-e1ac-444b-87a9-dcb8dffa2127 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:45.178634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:45.265157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5eae6-e1ac-444b-87a9-dcb8dffa2127 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:45.297639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447ce44-0e95-4fbc-8fae-43b20967b008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:45.298235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447ce44-0e95-4fbc-8fae-43b20967b008 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:45.298509 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447ce44-0e95-4fbc-8fae-43b20967b008 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:45.298779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447ce44-0e95-4fbc-8fae-43b20967b008 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:45.299420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447ce44-0e95-4fbc-8fae-43b20967b008 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:45.300863 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447ce44-0e95-4fbc-8fae-43b20967b008 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:45.326657 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 584/1173] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:20:45 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:20:47.235928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1290428-7c15-4c98-bd88-5f7ba8c7641c None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:47.236456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1290428-7c15-4c98-bd88-5f7ba8c7641c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:47.236456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1290428-7c15-4c98-bd88-5f7ba8c7641c None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:47.237850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 590/1174] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:47.243041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10be9a-c1b2-40cf-b20e-2cfaa59a4c5e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:47.243436 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10be9a-c1b2-40cf-b20e-2cfaa59a4c5e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:47.243594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10be9a-c1b2-40cf-b20e-2cfaa59a4c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:47.261364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ce10be9a-c1b2-40cf-b20e-2cfaa59a4c5e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:47.262534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ce10be9a-c1b2-40cf-b20e-2cfaa59a4c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:47.316831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:47.329078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 585/1175] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:47.596288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1290428-7c15-4c98-bd88-5f7ba8c7641c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:47.696328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1290428-7c15-4c98-bd88-5f7ba8c7641c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:47.708502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:47.729373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26beeb12-c957-45ab-9b6e-0d2eb4f4484a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:47.729995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beeb12-c957-45ab-9b6e-0d2eb4f4484a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:47.730276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beeb12-c957-45ab-9b6e-0d2eb4f4484a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:47.730550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beeb12-c957-45ab-9b6e-0d2eb4f4484a None admin] PATH_INFO: `/v3/domains/83dff104af5747c495b49785090514d5` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:47.731664 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26beeb12-c957-45ab-9b6e-0d2eb4f4484a None admin] RBAC: Authorizing `identity:get_domain(domain_id=83dff104af5747c495b49785090514d5)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:47.733134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26beeb12-c957-45ab-9b6e-0d2eb4f4484a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:47.735265 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 591/1176] 10.222.0.55 () {56 vars in 1183 bytes} [Fri Aug 19 16:20:47 2022] GET /identity/v3/domains/83dff104af5747c495b49785090514d5 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:47.741009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10be9a-c1b2-40cf-b20e-2cfaa59a4c5e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:47.835536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10be9a-c1b2-40cf-b20e-2cfaa59a4c5e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:47.868575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5689f1-d6d0-4d72-8420-e64b37c145c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:47.869135 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5689f1-d6d0-4d72-8420-e64b37c145c2 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:47.869410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5689f1-d6d0-4d72-8420-e64b37c145c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:47.869677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5689f1-d6d0-4d72-8420-e64b37c145c2 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:47.870303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5689f1-d6d0-4d72-8420-e64b37c145c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:47.871707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5689f1-d6d0-4d72-8420-e64b37c145c2 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:47.892306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 586/1177] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:20:47 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:20:49.814560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6bca8-8e72-4a6a-a267-c158477d424a None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:49.815136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6bca8-8e72-4a6a-a267-c158477d424a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:49.815136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6bca8-8e72-4a6a-a267-c158477d424a None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:49.816500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 592/1178] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:49.823028 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e65866-74dc-4c2f-af13-39d7fb22bdd7 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:49.823461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e65866-74dc-4c2f-af13-39d7fb22bdd7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:49.823950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e65866-74dc-4c2f-af13-39d7fb22bdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:49.847140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-71e65866-74dc-4c2f-af13-39d7fb22bdd7 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:49.848347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-71e65866-74dc-4c2f-af13-39d7fb22bdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:49.898212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:49.930125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 587/1179] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:50.228200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6bca8-8e72-4a6a-a267-c158477d424a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:50.353729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6bca8-8e72-4a6a-a267-c158477d424a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:50.387993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843a2d4-5394-4b10-99ad-cf21f114ba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:50.388613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843a2d4-5394-4b10-99ad-cf21f114ba1a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:50.388921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843a2d4-5394-4b10-99ad-cf21f114ba1a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:50.389192 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843a2d4-5394-4b10-99ad-cf21f114ba1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:50.399448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843a2d4-5394-4b10-99ad-cf21f114ba1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-b843a2d4-5394-4b10-99ad-cf21f114ba1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:20:50.404036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:50.405207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:50.406696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 593/1180] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:20:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:50.412071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e65866-74dc-4c2f-af13-39d7fb22bdd7 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:50.508093 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e65866-74dc-4c2f-af13-39d7fb22bdd7 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:50.540960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38ecb9-b66d-4acf-ae5d-c2f6d01cea35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:50.541563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38ecb9-b66d-4acf-ae5d-c2f6d01cea35 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:50.541840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38ecb9-b66d-4acf-ae5d-c2f6d01cea35 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:50.542108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38ecb9-b66d-4acf-ae5d-c2f6d01cea35 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:50.542756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38ecb9-b66d-4acf-ae5d-c2f6d01cea35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:50.543053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38ecb9-b66d-4acf-ae5d-c2f6d01cea35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:50.544411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38ecb9-b66d-4acf-ae5d-c2f6d01cea35 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:50.552916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 588/1181] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:20:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:50.558696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843a2d4-5394-4b10-99ad-cf21f114ba1a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:50.599517 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:50.656843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843a2d4-5394-4b10-99ad-cf21f114ba1a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:50.689560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a1c1a-d3c4-4325-800e-f6be406d38da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:50.690178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a1c1a-d3c4-4325-800e-f6be406d38da None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:50.690456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a1c1a-d3c4-4325-800e-f6be406d38da None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:50.690727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a1c1a-d3c4-4325-800e-f6be406d38da None admin] PATH_INFO: `/v3/users/87c28b0bdaa045448aeb1ef033ed8b73` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:50.703866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a1c1a-d3c4-4325-800e-f6be406d38da None admin] RBAC: Authorizing `identity:get_user(user_id=87c28b0bdaa045448aeb1ef033ed8b73)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:50.705347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a1c1a-d3c4-4325-800e-f6be406d38da None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:50.707001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 594/1182] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:50 2022] GET /identity/v3/users/87c28b0bdaa045448aeb1ef033ed8b73 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:50.712868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38ecb9-b66d-4acf-ae5d-c2f6d01cea35 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:50.752054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:50.809912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38ecb9-b66d-4acf-ae5d-c2f6d01cea35 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:50.842068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6bf05-eb01-41ac-8e02-238ee6c9cc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:50.842686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6bf05-eb01-41ac-8e02-238ee6c9cc0e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:50.842938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6bf05-eb01-41ac-8e02-238ee6c9cc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:50.843210 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6bf05-eb01-41ac-8e02-238ee6c9cc0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:50.909295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6bf05-eb01-41ac-8e02-238ee6c9cc0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:50.910738 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6bf05-eb01-41ac-8e02-238ee6c9cc0e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:50.946656 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:51.005696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 589/1183] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:20:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:20:51.011620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a1c1a-d3c4-4325-800e-f6be406d38da None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:51.106606 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:51.111024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a1c1a-d3c4-4325-800e-f6be406d38da None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:51.144843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbad31-212b-4c9c-bd05-9f38e74437f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:51.145465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbad31-212b-4c9c-bd05-9f38e74437f1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:51.145744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbad31-212b-4c9c-bd05-9f38e74437f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:51.146016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbad31-212b-4c9c-bd05-9f38e74437f1 None admin] PATH_INFO: `/v3/projects/c86bd275fd744a77850847c19dd21dfd` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:51.147123 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbad31-212b-4c9c-bd05-9f38e74437f1 None admin] RBAC: Authorizing `identity:get_project(project_id=c86bd275fd744a77850847c19dd21dfd)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:51.148542 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbad31-212b-4c9c-bd05-9f38e74437f1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:51.150196 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 595/1184] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:20:51 2022] GET /identity/v3/projects/c86bd275fd744a77850847c19dd21dfd => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:51.156469 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6bf05-eb01-41ac-8e02-238ee6c9cc0e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:51.251307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6bf05-eb01-41ac-8e02-238ee6c9cc0e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:51.291978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178d6b0-51be-4a7f-88c6-31756aa04357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:51.292563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178d6b0-51be-4a7f-88c6-31756aa04357 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:51.292868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178d6b0-51be-4a7f-88c6-31756aa04357 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:51.293140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178d6b0-51be-4a7f-88c6-31756aa04357 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:51.293848 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178d6b0-51be-4a7f-88c6-31756aa04357 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=c86bd275fd744a77850847c19dd21dfd, user.id=87c28b0bdaa045448aeb1ef033ed8b73) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:51.294149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178d6b0-51be-4a7f-88c6-31756aa04357 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:51.295564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178d6b0-51be-4a7f-88c6-31756aa04357 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:51.300779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 590/1185] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:20:51 2022] GET /identity/v3/role_assignments?user.id=87c28b0bdaa045448aeb1ef033ed8b73&scope.project.id=c86bd275fd744a77850847c19dd21dfd&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:20:53.150643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1c86c-e31b-4c4d-a1bf-ea3b890e52a4 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:53.151187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1c86c-e31b-4c4d-a1bf-ea3b890e52a4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:53.151187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1c86c-e31b-4c4d-a1bf-ea3b890e52a4 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:53.152504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 596/1186] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:53.157721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ebfe-5cc0-4635-a4ad-5cfa9bf569c6 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:53.158129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ebfe-5cc0-4635-a4ad-5cfa9bf569c6 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:53.158275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ebfe-5cc0-4635-a4ad-5cfa9bf569c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:53.164473 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:53.177423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-18e5ebfe-5cc0-4635-a4ad-5cfa9bf569c6 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:53.178608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-18e5ebfe-5cc0-4635-a4ad-5cfa9bf569c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:53.243538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 591/1187] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:53.509418 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1c86c-e31b-4c4d-a1bf-ea3b890e52a4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:53.607737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1c86c-e31b-4c4d-a1bf-ea3b890e52a4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:53.641316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e94fa-3d96-40b8-b3a6-3deffaf8c61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:53.641941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e94fa-3d96-40b8-b3a6-3deffaf8c61d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:53.642220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e94fa-3d96-40b8-b3a6-3deffaf8c61d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:53.642493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e94fa-3d96-40b8-b3a6-3deffaf8c61d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:53.652522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e94fa-3d96-40b8-b3a6-3deffaf8c61d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-8e3e94fa-3d96-40b8-b3a6-3deffaf8c61d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:20:53.657842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:53.658937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:20:53.660080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 597/1188] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:20:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:53.664866 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5ebfe-5cc0-4635-a4ad-5cfa9bf569c6 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:53.761047 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5ebfe-5cc0-4635-a4ad-5cfa9bf569c6 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:53.790194 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:53.795429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d98fd2-eb18-49bf-b0f0-73dccb0d6bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:53.796025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d98fd2-eb18-49bf-b0f0-73dccb0d6bad None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:53.796299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d98fd2-eb18-49bf-b0f0-73dccb0d6bad None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:53.796566 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d98fd2-eb18-49bf-b0f0-73dccb0d6bad None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:53.797249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d98fd2-eb18-49bf-b0f0-73dccb0d6bad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:53.797547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d98fd2-eb18-49bf-b0f0-73dccb0d6bad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:53.798894 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d98fd2-eb18-49bf-b0f0-73dccb0d6bad None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:53.806794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 592/1189] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:20:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:53.812668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e94fa-3d96-40b8-b3a6-3deffaf8c61d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:53.840205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:53.910660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e94fa-3d96-40b8-b3a6-3deffaf8c61d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:53.942121 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0aa68-08dd-49bf-8844-b8fe215ffcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:53.942731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0aa68-08dd-49bf-8844-b8fe215ffcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:53.943005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0aa68-08dd-49bf-8844-b8fe215ffcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:53.943275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0aa68-08dd-49bf-8844-b8fe215ffcb7 None admin] PATH_INFO: `/v3/users/87c28b0bdaa045448aeb1ef033ed8b73` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:53.944368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0aa68-08dd-49bf-8844-b8fe215ffcb7 None admin] RBAC: Authorizing `identity:get_user(user_id=87c28b0bdaa045448aeb1ef033ed8b73)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:53.945887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0aa68-08dd-49bf-8844-b8fe215ffcb7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:53.947430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 598/1190] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:53 2022] GET /identity/v3/users/87c28b0bdaa045448aeb1ef033ed8b73 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:53.953870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d98fd2-eb18-49bf-b0f0-73dccb0d6bad None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:54.050645 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d98fd2-eb18-49bf-b0f0-73dccb0d6bad None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:54.083019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112021e2-3c06-4af7-bc6c-7dca48badbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:54.083628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112021e2-3c06-4af7-bc6c-7dca48badbc8 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:54.083989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112021e2-3c06-4af7-bc6c-7dca48badbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:54.084263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112021e2-3c06-4af7-bc6c-7dca48badbc8 None admin] PATH_INFO: `/v3/projects/c86bd275fd744a77850847c19dd21dfd` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:54.085372 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112021e2-3c06-4af7-bc6c-7dca48badbc8 None admin] RBAC: Authorizing `identity:get_project(project_id=c86bd275fd744a77850847c19dd21dfd)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:54.086772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112021e2-3c06-4af7-bc6c-7dca48badbc8 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:54.088378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 593/1191] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:20:53 2022] GET /identity/v3/projects/c86bd275fd744a77850847c19dd21dfd => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:54.094488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0aa68-08dd-49bf-8844-b8fe215ffcb7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:54.185706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:54.193341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0aa68-08dd-49bf-8844-b8fe215ffcb7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:54.226656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411d083-7ada-4fc5-a2f2-bf411483d75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:54.227310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411d083-7ada-4fc5-a2f2-bf411483d75b None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:54.227592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411d083-7ada-4fc5-a2f2-bf411483d75b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:54.227869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411d083-7ada-4fc5-a2f2-bf411483d75b None admin] PATH_INFO: `/v3/projects/c86bd275fd744a77850847c19dd21dfd/users/87c28b0bdaa045448aeb1ef033ed8b73/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:54.229788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411d083-7ada-4fc5-a2f2-bf411483d75b None admin] RBAC: Authorizing `identity:create_grant(project_id=c86bd275fd744a77850847c19dd21dfd, user_id=87c28b0bdaa045448aeb1ef033ed8b73, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:54.231221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411d083-7ada-4fc5-a2f2-bf411483d75b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:54.243146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 599/1192] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:20:54 2022] PUT /identity/v3/projects/c86bd275fd744a77850847c19dd21dfd/users/87c28b0bdaa045448aeb1ef033ed8b73/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:20:56.115565 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df666e-d583-4cba-bfda-a178af1856bd None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:56.116086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df666e-d583-4cba-bfda-a178af1856bd None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:56.116086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df666e-d583-4cba-bfda-a178af1856bd None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:56.117464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 594/1193] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:20:56.122638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008c1a7-dd44-427b-a699-55637a61f2c8 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:56.123072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008c1a7-dd44-427b-a699-55637a61f2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:56.123200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008c1a7-dd44-427b-a699-55637a61f2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:56.141585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6008c1a7-dd44-427b-a699-55637a61f2c8 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:20:56.142782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6008c1a7-dd44-427b-a699-55637a61f2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:20:56.207334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 600/1194] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:20:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:20:56.471757 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df666e-d583-4cba-bfda-a178af1856bd None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:56.532939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:56.573036 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df666e-d583-4cba-bfda-a178af1856bd None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:56.605150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac0a73-8f43-4fa3-8cc8-498560ac48c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:56.605761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac0a73-8f43-4fa3-8cc8-498560ac48c4 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:56.606037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac0a73-8f43-4fa3-8cc8-498560ac48c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:56.606305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac0a73-8f43-4fa3-8cc8-498560ac48c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:56.617011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac0a73-8f43-4fa3-8cc8-498560ac48c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-c2ac0a73-8f43-4fa3-8cc8-498560ac48c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:20:56.621641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:20:56.622693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:20:56.623558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 595/1195] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:20:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:20:56.628944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008c1a7-dd44-427b-a699-55637a61f2c8 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:56.710525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:56.727867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008c1a7-dd44-427b-a699-55637a61f2c8 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:56.759958 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d92ea-ba6b-46aa-b6da-68c2a6be4d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:56.760562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d92ea-ba6b-46aa-b6da-68c2a6be4d1b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:56.760871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d92ea-ba6b-46aa-b6da-68c2a6be4d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:56.761145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d92ea-ba6b-46aa-b6da-68c2a6be4d1b None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:56.761807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d92ea-ba6b-46aa-b6da-68c2a6be4d1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:56.762112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d92ea-ba6b-46aa-b6da-68c2a6be4d1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:56.763475 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d92ea-ba6b-46aa-b6da-68c2a6be4d1b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:56.771723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 601/1196] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:20:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:56.777538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac0a73-8f43-4fa3-8cc8-498560ac48c4 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:56.873050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac0a73-8f43-4fa3-8cc8-498560ac48c4 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:56.894724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:56.906759 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb92ca-fd38-480a-8071-8af01c410297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:56.907371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb92ca-fd38-480a-8071-8af01c410297 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:56.907646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb92ca-fd38-480a-8071-8af01c410297 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:56.907915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb92ca-fd38-480a-8071-8af01c410297 None admin] PATH_INFO: `/v3/users/87c28b0bdaa045448aeb1ef033ed8b73` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:56.909040 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb92ca-fd38-480a-8071-8af01c410297 None admin] RBAC: Authorizing `identity:get_user(user_id=87c28b0bdaa045448aeb1ef033ed8b73)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:56.910464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb92ca-fd38-480a-8071-8af01c410297 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:56.911998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 596/1197] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Aug 19 16:20:56 2022] GET /identity/v3/users/87c28b0bdaa045448aeb1ef033ed8b73 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:56.917823 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d92ea-ba6b-46aa-b6da-68c2a6be4d1b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:57.014246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d92ea-ba6b-46aa-b6da-68c2a6be4d1b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:57.047145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486160d4-fefc-4a09-b869-fbb4f09534a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:57.047768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486160d4-fefc-4a09-b869-fbb4f09534a5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:57.048051 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486160d4-fefc-4a09-b869-fbb4f09534a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:57.048326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486160d4-fefc-4a09-b869-fbb4f09534a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:57.063648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:57.115184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486160d4-fefc-4a09-b869-fbb4f09534a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:57.116627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486160d4-fefc-4a09-b869-fbb4f09534a5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:57.212957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 602/1198] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:20:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:20:57.218967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb92ca-fd38-480a-8071-8af01c410297 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:57.315673 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb92ca-fd38-480a-8071-8af01c410297 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:57.347589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a51ac4-bf26-48e4-8bae-79e46b93b1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:57.348196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a51ac4-bf26-48e4-8bae-79e46b93b1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:57.348471 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a51ac4-bf26-48e4-8bae-79e46b93b1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:57.349017 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a51ac4-bf26-48e4-8bae-79e46b93b1d4 None admin] PATH_INFO: `/v3/projects/c86bd275fd744a77850847c19dd21dfd` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:57.350109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a51ac4-bf26-48e4-8bae-79e46b93b1d4 None admin] RBAC: Authorizing `identity:get_project(project_id=c86bd275fd744a77850847c19dd21dfd)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:57.351527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a51ac4-bf26-48e4-8bae-79e46b93b1d4 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:57.353158 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 597/1199] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Aug 19 16:20:57 2022] GET /identity/v3/projects/c86bd275fd744a77850847c19dd21dfd => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:20:57.359213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486160d4-fefc-4a09-b869-fbb4f09534a5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:20:57.404527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:20:57.456072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486160d4-fefc-4a09-b869-fbb4f09534a5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:20:57.488962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572236ac-ba1a-4d45-ba21-d8abeba92ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:20:57.489551 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572236ac-ba1a-4d45-ba21-d8abeba92ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:20:57.489827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572236ac-ba1a-4d45-ba21-d8abeba92ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:20:57.490097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572236ac-ba1a-4d45-ba21-d8abeba92ee0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:20:57.490808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572236ac-ba1a-4d45-ba21-d8abeba92ee0 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=c86bd275fd744a77850847c19dd21dfd, user.id=87c28b0bdaa045448aeb1ef033ed8b73) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:20:57.491113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572236ac-ba1a-4d45-ba21-d8abeba92ee0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:20:57.492521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572236ac-ba1a-4d45-ba21-d8abeba92ee0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:20:57.498027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 603/1200] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:20:57 2022] GET /identity/v3/role_assignments?user.id=87c28b0bdaa045448aeb1ef033ed8b73&scope.project.id=c86bd275fd744a77850847c19dd21dfd&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:21:07.343568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=71 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:07.495520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=45 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:52.519827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fb258-649a-46d6-92c3-54d46a7907a9 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:52.520487 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fb258-649a-46d6-92c3-54d46a7907a9 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:52.520581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fb258-649a-46d6-92c3-54d46a7907a9 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:52.522069 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 598/1201] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:21:52.528389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce7655-4658-47f4-ac3c-9f9674b3b049 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:52.528981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce7655-4658-47f4-ac3c-9f9674b3b049 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:52.528981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce7655-4658-47f4-ac3c-9f9674b3b049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:52.549930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c3ce7655-4658-47f4-ac3c-9f9674b3b049 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:21:52.551305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c3ce7655-4658-47f4-ac3c-9f9674b3b049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:21:52.647059 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 604/1202] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:21:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:21:52.939091 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fb258-649a-46d6-92c3-54d46a7907a9 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:53.083070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fb258-649a-46d6-92c3-54d46a7907a9 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:53.135609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b0dfe-4b83-46ff-b0e1-f28d2e186af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:53.136348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b0dfe-4b83-46ff-b0e1-f28d2e186af3 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:53.136642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b0dfe-4b83-46ff-b0e1-f28d2e186af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:53.137596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b0dfe-4b83-46ff-b0e1-f28d2e186af3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:53.150225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b0dfe-4b83-46ff-b0e1-f28d2e186af3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:53.151784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b0dfe-4b83-46ff-b0e1-f28d2e186af3 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-446b0dfe-4b83-46ff-b0e1-f28d2e186af3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:21:53.169612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:21:53.170739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:21:53.173960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:21:53.173960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:21:53.173960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:21:53.173960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 599/1203] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:21:53.177182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce7655-4658-47f4-ac3c-9f9674b3b049 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:53.337441 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce7655-4658-47f4-ac3c-9f9674b3b049 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:53.370511 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:53.379087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702d48c-9d32-4265-8529-4a277f489f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:53.379762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702d48c-9d32-4265-8529-4a277f489f25 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:53.380038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702d48c-9d32-4265-8529-4a277f489f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:53.380345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702d48c-9d32-4265-8529-4a277f489f25 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:53.381240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702d48c-9d32-4265-8529-4a277f489f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:21:53.381605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702d48c-9d32-4265-8529-4a277f489f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:53.383240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702d48c-9d32-4265-8529-4a277f489f25 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:53.404793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 605/1204] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:21:53.407157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b0dfe-4b83-46ff-b0e1-f28d2e186af3 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:53.485356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:53.545199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b0dfe-4b83-46ff-b0e1-f28d2e186af3 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:53.588120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8584e-0f7c-4e7a-8b9a-58cff39a83e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:53.588653 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8584e-0f7c-4e7a-8b9a-58cff39a83e8 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:53.588979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8584e-0f7c-4e7a-8b9a-58cff39a83e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:53.589262 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8584e-0f7c-4e7a-8b9a-58cff39a83e8 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:53.590062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8584e-0f7c-4e7a-8b9a-58cff39a83e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:53.591846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8584e-0f7c-4e7a-8b9a-58cff39a83e8 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:53.631736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 600/1205] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Aug 19 16:21:53 2022] POST /identity/v3/users => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:21:55.754974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8668e-4f7f-438b-9069-9d434a8241e0 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:55.755521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8668e-4f7f-438b-9069-9d434a8241e0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:55.755521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8668e-4f7f-438b-9069-9d434a8241e0 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:55.756962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 606/1206] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:21:55.765828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0501a7d-7e52-4221-94a4-f6bf191b5530 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:55.765828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0501a7d-7e52-4221-94a4-f6bf191b5530 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:55.765828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0501a7d-7e52-4221-94a4-f6bf191b5530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:55.783054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-e0501a7d-7e52-4221-94a4-f6bf191b5530 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:21:55.784354 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e0501a7d-7e52-4221-94a4-f6bf191b5530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:21:55.859147 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 601/1207] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:21:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:21:56.141086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8668e-4f7f-438b-9069-9d434a8241e0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:56.274119 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8668e-4f7f-438b-9069-9d434a8241e0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:56.315389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2ee98-99b0-4265-8327-85c2eb843f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:56.316509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2ee98-99b0-4265-8327-85c2eb843f16 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:56.316952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2ee98-99b0-4265-8327-85c2eb843f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:56.317228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2ee98-99b0-4265-8327-85c2eb843f16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:56.331046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2ee98-99b0-4265-8327-85c2eb843f16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-22c2ee98-99b0-4265-8327-85c2eb843f16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:21:56.336621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:21:56.337674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:21:56.338500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:21:56.338500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:21:56.338500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:21:56.338500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:21:56.338500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:21:56.338500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:21:56.338500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:21:56.338500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:21:56.338500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:21:56.338500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:21:56.338500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:21:56.338500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:21:56.342203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 607/1208] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:21:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:21:56.346222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0501a7d-7e52-4221-94a4-f6bf191b5530 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:56.388788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:56.455334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0501a7d-7e52-4221-94a4-f6bf191b5530 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:56.493815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31fa78-b657-416a-b053-464109a5038d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:56.494479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31fa78-b657-416a-b053-464109a5038d None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:56.494750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31fa78-b657-416a-b053-464109a5038d None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:56.495021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31fa78-b657-416a-b053-464109a5038d None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:56.495689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31fa78-b657-416a-b053-464109a5038d None admin] RBAC: Adding query filter params (name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:21:56.496539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31fa78-b657-416a-b053-464109a5038d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:56.498130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31fa78-b657-416a-b053-464109a5038d None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:56.507832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 602/1209] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:21:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:21:56.514372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2ee98-99b0-4265-8327-85c2eb843f16 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:56.576433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:56.640809 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2ee98-99b0-4265-8327-85c2eb843f16 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:56.688685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb099f-7f17-474b-9009-b5a5bcd52452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:56.689574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb099f-7f17-474b-9009-b5a5bcd52452 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:56.689900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb099f-7f17-474b-9009-b5a5bcd52452 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:56.690253 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb099f-7f17-474b-9009-b5a5bcd52452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:56.702994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb099f-7f17-474b-9009-b5a5bcd52452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:56.704577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb099f-7f17-474b-9009-b5a5bcd52452 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-e3cb099f-7f17-474b-9009-b5a5bcd52452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:21:56.720099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:21:56.721568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:21:56.722563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:21:56.722563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:21:56.722563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:21:56.726584 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 608/1210] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:21:56.727959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31fa78-b657-416a-b053-464109a5038d None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:56.841111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31fa78-b657-416a-b053-464109a5038d None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:56.842807 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:56.883712 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfc337-7138-4d9d-b432-87b33ab9f6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:56.884419 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfc337-7138-4d9d-b432-87b33ab9f6ea None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:56.884710 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfc337-7138-4d9d-b432-87b33ab9f6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:56.885515 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfc337-7138-4d9d-b432-87b33ab9f6ea None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:56.885765 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfc337-7138-4d9d-b432-87b33ab9f6ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:21:56.886044 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfc337-7138-4d9d-b432-87b33ab9f6ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:56.887616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfc337-7138-4d9d-b432-87b33ab9f6ea None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:56.902919 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 603/1211] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:21:56.909085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb099f-7f17-474b-9009-b5a5bcd52452 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:57.033536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb099f-7f17-474b-9009-b5a5bcd52452 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:57.035279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:57.077409 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236396ea-58ca-4dc2-889f-567f86b85e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:57.078203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236396ea-58ca-4dc2-889f-567f86b85e08 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:57.078492 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236396ea-58ca-4dc2-889f-567f86b85e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:57.078768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236396ea-58ca-4dc2-889f-567f86b85e08 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:57.089354 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236396ea-58ca-4dc2-889f-567f86b85e08 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:57.091764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236396ea-58ca-4dc2-889f-567f86b85e08 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-236396ea-58ca-4dc2-889f-567f86b85e08 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:21:57.107835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:21:57.109112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:21:57.110143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:21:57.110143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:21:57.110143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:21:57.110143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:21:57.110143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:21:57.110143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:21:57.110143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:21:57.110143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:21:57.110143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 16:21:57.110143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:21:57.115126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 609/1212] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:21:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:21:57.116826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfc337-7138-4d9d-b432-87b33ab9f6ea None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:57.238329 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfc337-7138-4d9d-b432-87b33ab9f6ea None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:57.277941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9975060-3c33-4ecc-9497-90a2006e131a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:57.278541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9975060-3c33-4ecc-9497-90a2006e131a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:57.278804 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9975060-3c33-4ecc-9497-90a2006e131a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:57.279151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9975060-3c33-4ecc-9497-90a2006e131a None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:57.279942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9975060-3c33-4ecc-9497-90a2006e131a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:21:57.280318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9975060-3c33-4ecc-9497-90a2006e131a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:57.281989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9975060-3c33-4ecc-9497-90a2006e131a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:57.305096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 604/1213] 10.222.0.55 () {56 vars in 1143 bytes} [Fri Aug 19 16:21:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:21:57.311566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236396ea-58ca-4dc2-889f-567f86b85e08 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:57.435452 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236396ea-58ca-4dc2-889f-567f86b85e08 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:57.479952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fed4b-0093-46d0-8123-0a9fca852741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:57.480595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fed4b-0093-46d0-8123-0a9fca852741 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:57.480899 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fed4b-0093-46d0-8123-0a9fca852741 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:57.481175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fed4b-0093-46d0-8123-0a9fca852741 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:57.521545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:57.565623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fed4b-0093-46d0-8123-0a9fca852741 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:57.566936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fed4b-0093-46d0-8123-0a9fca852741 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:57.693992 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 610/1214] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:21:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 16:21:57.701139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9975060-3c33-4ecc-9497-90a2006e131a None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:57.723927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:57.827201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9975060-3c33-4ecc-9497-90a2006e131a None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:57.864969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fa626-7a9d-4436-89d8-a11a20e88998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:57.865792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fa626-7a9d-4436-89d8-a11a20e88998 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:57.865968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fa626-7a9d-4436-89d8-a11a20e88998 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:57.866250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fa626-7a9d-4436-89d8-a11a20e88998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:57.877577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fa626-7a9d-4436-89d8-a11a20e88998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:57.879033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fa626-7a9d-4436-89d8-a11a20e88998 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-c10fa626-7a9d-4436-89d8-a11a20e88998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:21:57.902466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:21:57.903761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:21:57.904880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:21:57.904880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:21:57.904880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:21:57.904880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 605/1215] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:21:57.907530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fed4b-0093-46d0-8123-0a9fca852741 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:57.982198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:58.029988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fed4b-0093-46d0-8123-0a9fca852741 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:58.070832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdeff62-cb33-475e-bac0-be62471d4db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:58.071513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdeff62-cb33-475e-bac0-be62471d4db0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:58.071889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdeff62-cb33-475e-bac0-be62471d4db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:58.072250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdeff62-cb33-475e-bac0-be62471d4db0 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:58.072994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdeff62-cb33-475e-bac0-be62471d4db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:21:58.073395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdeff62-cb33-475e-bac0-be62471d4db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:58.075538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdeff62-cb33-475e-bac0-be62471d4db0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:58.089916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 611/1216] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:21:58.096209 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fa626-7a9d-4436-89d8-a11a20e88998 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:58.190917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:58.225561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fa626-7a9d-4436-89d8-a11a20e88998 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:58.262856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a0750-8800-4e27-9c3b-42f0131b34d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:58.263482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a0750-8800-4e27-9c3b-42f0131b34d2 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:58.263778 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a0750-8800-4e27-9c3b-42f0131b34d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:58.264036 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a0750-8800-4e27-9c3b-42f0131b34d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:58.271345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a0750-8800-4e27-9c3b-42f0131b34d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:58.272808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a0750-8800-4e27-9c3b-42f0131b34d2 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-598a0750-8800-4e27-9c3b-42f0131b34d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:21:58.283887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:21:58.284904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:21:58.291012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:21:58.291012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:21:58.291012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:21:58.291012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:21:58.291012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:21:58.291012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:21:58.291012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:21:58.291012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:21:58.291012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 606/1217] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:21:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:21:58.291537 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdeff62-cb33-475e-bac0-be62471d4db0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:58.413406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdeff62-cb33-475e-bac0-be62471d4db0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:58.442766 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:58.457464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5686f-7056-43a7-b077-6132386f983c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:58.458189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5686f-7056-43a7-b077-6132386f983c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:58.458532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5686f-7056-43a7-b077-6132386f983c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:58.458804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5686f-7056-43a7-b077-6132386f983c None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:58.459641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5686f-7056-43a7-b077-6132386f983c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:21:58.460064 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5686f-7056-43a7-b077-6132386f983c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:58.461531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5686f-7056-43a7-b077-6132386f983c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:58.482340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 612/1218] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:21:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:21:58.482548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a0750-8800-4e27-9c3b-42f0131b34d2 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:21:58.593816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a0750-8800-4e27-9c3b-42f0131b34d2 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:21:58.627633 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:21:58.635267 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b973dc-575f-468f-983a-c9cad136ecac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:21:58.635985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b973dc-575f-468f-983a-c9cad136ecac None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:21:58.636378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b973dc-575f-468f-983a-c9cad136ecac None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:21:58.636804 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b973dc-575f-468f-983a-c9cad136ecac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:21:58.637879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b973dc-575f-468f-983a-c9cad136ecac None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=1dcbe2b128a94b9abbf3a9b25db1e845) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:21:58.638205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b973dc-575f-468f-983a-c9cad136ecac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:21:58.639798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b973dc-575f-468f-983a-c9cad136ecac None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:21:58.646362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 607/1219] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:21:58 2022] GET /identity/v3/role_assignments?user.id=1dcbe2b128a94b9abbf3a9b25db1e845&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:00.693084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5541091-d89a-467f-bbe8-b03c57f68317 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:00.693626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5541091-d89a-467f-bbe8-b03c57f68317 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:00.693785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5541091-d89a-467f-bbe8-b03c57f68317 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:00.695054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 613/1220] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:22:00.702830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa74be-aab9-492b-a55a-0a101872dd90 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:00.703259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa74be-aab9-492b-a55a-0a101872dd90 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:00.703463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa74be-aab9-492b-a55a-0a101872dd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:00.722737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-28aa74be-aab9-492b-a55a-0a101872dd90 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:22:00.723926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-28aa74be-aab9-492b-a55a-0a101872dd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:22:00.798790 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 608/1221] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:22:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:22:01.102799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5541091-d89a-467f-bbe8-b03c57f68317 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:01.217130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5541091-d89a-467f-bbe8-b03c57f68317 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:01.257445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f3f1b-319f-407a-8dd5-c831fb9708ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:01.258159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f3f1b-319f-407a-8dd5-c831fb9708ab None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:01.258472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f3f1b-319f-407a-8dd5-c831fb9708ab None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:01.258900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f3f1b-319f-407a-8dd5-c831fb9708ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:01.269912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f3f1b-319f-407a-8dd5-c831fb9708ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-1d2f3f1b-319f-407a-8dd5-c831fb9708ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:22:01.274577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:01.275677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:01.276571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 614/1222] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:22:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:22:01.281941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa74be-aab9-492b-a55a-0a101872dd90 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:01.391699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa74be-aab9-492b-a55a-0a101872dd90 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:01.416260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:01.428150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaf682-c4ee-4e49-a17d-5bd4824bb756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:01.428844 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaf682-c4ee-4e49-a17d-5bd4824bb756 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:01.429112 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaf682-c4ee-4e49-a17d-5bd4824bb756 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:01.429388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaf682-c4ee-4e49-a17d-5bd4824bb756 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:01.430074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eaf682-c4ee-4e49-a17d-5bd4824bb756 None admin] RBAC: Adding query filter params (name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:01.430372 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eaf682-c4ee-4e49-a17d-5bd4824bb756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:01.431729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eaf682-c4ee-4e49-a17d-5bd4824bb756 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:01.440509 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 609/1223] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:22:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:01.447145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f3f1b-319f-407a-8dd5-c831fb9708ab None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:01.493701 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:01.563727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f3f1b-319f-407a-8dd5-c831fb9708ab None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:01.598222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901629b-a2bd-46de-a3af-fddf30f10475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:01.598859 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901629b-a2bd-46de-a3af-fddf30f10475 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:01.599143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901629b-a2bd-46de-a3af-fddf30f10475 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:01.599422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901629b-a2bd-46de-a3af-fddf30f10475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:01.611840 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901629b-a2bd-46de-a3af-fddf30f10475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:01.613555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901629b-a2bd-46de-a3af-fddf30f10475 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-f901629b-a2bd-46de-a3af-fddf30f10475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:22:01.627965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:01.629134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:22:01.630159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:22:01.630159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:01.630159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:01.633208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 615/1224] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:01.635709 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaf682-c4ee-4e49-a17d-5bd4824bb756 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:01.755716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaf682-c4ee-4e49-a17d-5bd4824bb756 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:01.793822 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc51ac8-a7bb-43d7-b4f4-9af7914af1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:01.794425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc51ac8-a7bb-43d7-b4f4-9af7914af1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:01.794694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc51ac8-a7bb-43d7-b4f4-9af7914af1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:01.794964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc51ac8-a7bb-43d7-b4f4-9af7914af1f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:01.795619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc51ac8-a7bb-43d7-b4f4-9af7914af1f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:01.795896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc51ac8-a7bb-43d7-b4f4-9af7914af1f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:01.797727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc51ac8-a7bb-43d7-b4f4-9af7914af1f6 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:01.818397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 610/1225] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:22:01.824180 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901629b-a2bd-46de-a3af-fddf30f10475 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:01.927942 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:01.935295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901629b-a2bd-46de-a3af-fddf30f10475 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:01.974448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24e78b-a660-4f4c-b761-a87e3c4ae5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:01.975081 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24e78b-a660-4f4c-b761-a87e3c4ae5dc None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:01.975372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24e78b-a660-4f4c-b761-a87e3c4ae5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:01.975679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24e78b-a660-4f4c-b761-a87e3c4ae5dc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:01.983914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24e78b-a660-4f4c-b761-a87e3c4ae5dc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:01.985483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24e78b-a660-4f4c-b761-a87e3c4ae5dc None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-ca24e78b-a660-4f4c-b761-a87e3c4ae5dc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:22:01.997382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:01.998518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:01.999633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:01.999633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:01.999633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:22:01.999633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:22:01.999633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:22:01.999633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:22:01.999633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:22:01.999633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:22:01.999633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 16:22:01.999633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:01.999633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 616/1226] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:02.004588 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc51ac8-a7bb-43d7-b4f4-9af7914af1f6 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:02.059417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:02.122785 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc51ac8-a7bb-43d7-b4f4-9af7914af1f6 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:02.160053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90735189-d7ca-42cf-bfd6-709ecfe4877e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:02.160729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90735189-d7ca-42cf-bfd6-709ecfe4877e None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:02.161094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90735189-d7ca-42cf-bfd6-709ecfe4877e None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:02.161368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90735189-d7ca-42cf-bfd6-709ecfe4877e None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:02.162094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90735189-d7ca-42cf-bfd6-709ecfe4877e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:02.162459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90735189-d7ca-42cf-bfd6-709ecfe4877e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:02.163878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90735189-d7ca-42cf-bfd6-709ecfe4877e None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:02.181629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 611/1227] 10.222.0.55 () {56 vars in 1143 bytes} [Fri Aug 19 16:22:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:02.188006 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24e78b-a660-4f4c-b761-a87e3c4ae5dc None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:02.300285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24e78b-a660-4f4c-b761-a87e3c4ae5dc None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:02.341971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e16836-098a-4423-a86e-ba94f3ffc048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:02.342949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16836-098a-4423-a86e-ba94f3ffc048 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:02.343261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16836-098a-4423-a86e-ba94f3ffc048 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:02.343564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16836-098a-4423-a86e-ba94f3ffc048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:02.356618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e16836-098a-4423-a86e-ba94f3ffc048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:02.358155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e16836-098a-4423-a86e-ba94f3ffc048 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a5e16836-098a-4423-a86e-ba94f3ffc048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:22:02.373538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:02.374874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:22:02.376242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:22:02.376242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:02.376242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:02.376242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 617/1228] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:02.380740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90735189-d7ca-42cf-bfd6-709ecfe4877e None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:02.493937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90735189-d7ca-42cf-bfd6-709ecfe4877e None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:02.496900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:02.533270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa089267-7398-4aef-b25f-3d23b973faa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:02.533904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa089267-7398-4aef-b25f-3d23b973faa2 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:02.534185 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa089267-7398-4aef-b25f-3d23b973faa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:02.534461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa089267-7398-4aef-b25f-3d23b973faa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:02.535108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa089267-7398-4aef-b25f-3d23b973faa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:02.535415 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa089267-7398-4aef-b25f-3d23b973faa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:02.536825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa089267-7398-4aef-b25f-3d23b973faa2 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:02.549121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 612/1229] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:22:02.555936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e16836-098a-4423-a86e-ba94f3ffc048 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:02.566868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:02.670818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e16836-098a-4423-a86e-ba94f3ffc048 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:02.710058 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087d58c-fec6-44ce-9380-78f01d285a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:02.710770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087d58c-fec6-44ce-9380-78f01d285a7c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:02.711090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087d58c-fec6-44ce-9380-78f01d285a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:02.711371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087d58c-fec6-44ce-9380-78f01d285a7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:02.719228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087d58c-fec6-44ce-9380-78f01d285a7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:02.721332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087d58c-fec6-44ce-9380-78f01d285a7c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-e087d58c-fec6-44ce-9380-78f01d285a7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:22:02.734706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:02.735667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:02.736571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:22:02.736571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:22:02.736571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:22:02.736571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:22:02.736571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:22:02.736571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:22:02.736571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:22:02.736571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:02.736571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 618/1230] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:22:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:02.741680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa089267-7398-4aef-b25f-3d23b973faa2 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:02.856259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa089267-7398-4aef-b25f-3d23b973faa2 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:02.895141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfe717-d0d7-47a9-b59f-b45189cf4483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:02.895854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfe717-d0d7-47a9-b59f-b45189cf4483 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:02.896092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfe717-d0d7-47a9-b59f-b45189cf4483 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:02.896418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfe717-d0d7-47a9-b59f-b45189cf4483 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:02.897190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfe717-d0d7-47a9-b59f-b45189cf4483 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:02.897499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfe717-d0d7-47a9-b59f-b45189cf4483 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:02.898961 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfe717-d0d7-47a9-b59f-b45189cf4483 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:02.911716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 613/1231] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:22:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:02.918719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087d58c-fec6-44ce-9380-78f01d285a7c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:02.990576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:03.031300 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087d58c-fec6-44ce-9380-78f01d285a7c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:03.068294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ef9d7-6e8c-4001-81b4-48087a12cbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:03.068989 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ef9d7-6e8c-4001-81b4-48087a12cbec None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:03.069272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ef9d7-6e8c-4001-81b4-48087a12cbec None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:03.069554 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ef9d7-6e8c-4001-81b4-48087a12cbec None admin] PATH_INFO: `/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/1dcbe2b128a94b9abbf3a9b25db1e845/roles/3a9f17a4e0074293989007ce1a8b2565` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:03.084553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ef9d7-6e8c-4001-81b4-48087a12cbec None admin] RBAC: Authorizing `identity:create_grant(project_id=d143cf444f354ec0bf2089dae3d94255, user_id=1dcbe2b128a94b9abbf3a9b25db1e845, role_id=3a9f17a4e0074293989007ce1a8b2565)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:03.086233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ef9d7-6e8c-4001-81b4-48087a12cbec None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:03.101168 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 619/1232] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:22:02 2022] PUT /identity/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/1dcbe2b128a94b9abbf3a9b25db1e845/roles/3a9f17a4e0074293989007ce1a8b2565 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:22:05.078768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3365c-3399-4885-b755-7c7918608b7d None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:05.080869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3365c-3399-4885-b755-7c7918608b7d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:05.080869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3365c-3399-4885-b755-7c7918608b7d None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:05.080869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 614/1233] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:22:05.086773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08802f83-6442-4941-8bc8-c88fb7bb0e3c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:05.087301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08802f83-6442-4941-8bc8-c88fb7bb0e3c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:05.087419 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08802f83-6442-4941-8bc8-c88fb7bb0e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:05.108696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-08802f83-6442-4941-8bc8-c88fb7bb0e3c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:22:05.109996 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-08802f83-6442-4941-8bc8-c88fb7bb0e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:22:05.184835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 620/1234] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:22:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:22:05.494991 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3365c-3399-4885-b755-7c7918608b7d None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:05.525780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:05.610635 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3365c-3399-4885-b755-7c7918608b7d None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:05.649636 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6202a-73d0-477c-97df-c2972d99222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:05.650271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6202a-73d0-477c-97df-c2972d99222c None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:05.650549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6202a-73d0-477c-97df-c2972d99222c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:05.650852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6202a-73d0-477c-97df-c2972d99222c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:05.662598 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6202a-73d0-477c-97df-c2972d99222c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-7fe6202a-73d0-477c-97df-c2972d99222c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:22:05.668985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:05.670236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:05.671556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 615/1235] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Aug 19 16:22:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:05.676895 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08802f83-6442-4941-8bc8-c88fb7bb0e3c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:05.727976 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:05.787951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08802f83-6442-4941-8bc8-c88fb7bb0e3c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:05.829387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63359850-bed8-4eb6-ad64-1c37b4daf442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:05.830204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63359850-bed8-4eb6-ad64-1c37b4daf442 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:05.830509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63359850-bed8-4eb6-ad64-1c37b4daf442 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:05.830826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63359850-bed8-4eb6-ad64-1c37b4daf442 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:05.831584 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63359850-bed8-4eb6-ad64-1c37b4daf442 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:05.831893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63359850-bed8-4eb6-ad64-1c37b4daf442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:05.833455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63359850-bed8-4eb6-ad64-1c37b4daf442 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:05.847401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 621/1236] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Aug 19 16:22:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:05.848701 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6202a-73d0-477c-97df-c2972d99222c None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:05.943832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:05.961877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6202a-73d0-477c-97df-c2972d99222c None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:06.006974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573e201-1351-4d39-959c-febf7fadf5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:06.007609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573e201-1351-4d39-959c-febf7fadf5fe None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:06.007890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573e201-1351-4d39-959c-febf7fadf5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:06.008164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573e201-1351-4d39-959c-febf7fadf5fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:06.021599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573e201-1351-4d39-959c-febf7fadf5fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:06.023320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573e201-1351-4d39-959c-febf7fadf5fe None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-2573e201-1351-4d39-959c-febf7fadf5fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:22:06.040575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:06.041751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:22:06.043434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:22:06.043434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:06.043434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:06.043434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 616/1237] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:06.048704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63359850-bed8-4eb6-ad64-1c37b4daf442 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:06.168430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63359850-bed8-4eb6-ad64-1c37b4daf442 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:06.177817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:06.202994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a12b6c-ff31-45c2-a396-51c9e6242bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:06.203636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a12b6c-ff31-45c2-a396-51c9e6242bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:06.203888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a12b6c-ff31-45c2-a396-51c9e6242bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:06.204158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a12b6c-ff31-45c2-a396-51c9e6242bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:06.204829 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a12b6c-ff31-45c2-a396-51c9e6242bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:06.205135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a12b6c-ff31-45c2-a396-51c9e6242bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:06.206496 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a12b6c-ff31-45c2-a396-51c9e6242bb0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:06.220468 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 622/1238] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:06.226859 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573e201-1351-4d39-959c-febf7fadf5fe None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:06.341438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573e201-1351-4d39-959c-febf7fadf5fe None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:06.380115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8942280-bbb8-44e4-b6e0-ee62a57beb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:06.380877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8942280-bbb8-44e4-b6e0-ee62a57beb05 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:06.381158 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8942280-bbb8-44e4-b6e0-ee62a57beb05 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:06.381489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8942280-bbb8-44e4-b6e0-ee62a57beb05 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:06.384303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:06.391431 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8942280-bbb8-44e4-b6e0-ee62a57beb05 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:06.393118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8942280-bbb8-44e4-b6e0-ee62a57beb05 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-e8942280-bbb8-44e4-b6e0-ee62a57beb05 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:22:06.406181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:06.407366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:06.408504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:06.408504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:06.408504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:22:06.408504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:22:06.408504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:22:06.408504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:22:06.408504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:22:06.408504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:22:06.408504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 16:22:06.408504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:06.408504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 617/1239] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:22:06.413738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a12b6c-ff31-45c2-a396-51c9e6242bb0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:06.524029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a12b6c-ff31-45c2-a396-51c9e6242bb0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:06.560665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d9a61-714b-4ac2-9ae7-609ccf9b75e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:06.561559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d9a61-714b-4ac2-9ae7-609ccf9b75e6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:06.561837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d9a61-714b-4ac2-9ae7-609ccf9b75e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:06.562104 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d9a61-714b-4ac2-9ae7-609ccf9b75e6 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:06.562823 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d9a61-714b-4ac2-9ae7-609ccf9b75e6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:06.563133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d9a61-714b-4ac2-9ae7-609ccf9b75e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:06.564562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d9a61-714b-4ac2-9ae7-609ccf9b75e6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:06.582585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 623/1240] 10.222.0.55 () {56 vars in 1143 bytes} [Fri Aug 19 16:22:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:22:06.589469 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8942280-bbb8-44e4-b6e0-ee62a57beb05 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:06.705328 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8942280-bbb8-44e4-b6e0-ee62a57beb05 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:06.745097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faa249-9df6-4f07-b7c5-4dbed7bc4727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:06.745705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faa249-9df6-4f07-b7c5-4dbed7bc4727 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:06.745982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faa249-9df6-4f07-b7c5-4dbed7bc4727 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:06.746286 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faa249-9df6-4f07-b7c5-4dbed7bc4727 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:06.817803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:06.826580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02faa249-9df6-4f07-b7c5-4dbed7bc4727 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:06.828249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02faa249-9df6-4f07-b7c5-4dbed7bc4727 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:06.938998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 618/1241] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:22:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:22:06.945228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d9a61-714b-4ac2-9ae7-609ccf9b75e6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:06.977253 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:07.064761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d9a61-714b-4ac2-9ae7-609ccf9b75e6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:07.106518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9461fda-3462-4faf-8b99-2adacf03f029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:07.107311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9461fda-3462-4faf-8b99-2adacf03f029 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:07.107670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9461fda-3462-4faf-8b99-2adacf03f029 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:07.108019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9461fda-3462-4faf-8b99-2adacf03f029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:07.122052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9461fda-3462-4faf-8b99-2adacf03f029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:07.124020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9461fda-3462-4faf-8b99-2adacf03f029 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c9461fda-3462-4faf-8b99-2adacf03f029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:22:07.143481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:07.144672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:22:07.145597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:22:07.145597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:07.145597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:07.145597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 624/1242] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:07.150473 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faa249-9df6-4f07-b7c5-4dbed7bc4727 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:07.253798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:07.260726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faa249-9df6-4f07-b7c5-4dbed7bc4727 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:07.295559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93287aba-c705-4e25-bc5f-ee599e5866e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:07.296165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93287aba-c705-4e25-bc5f-ee599e5866e9 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:07.296454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93287aba-c705-4e25-bc5f-ee599e5866e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:07.296727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93287aba-c705-4e25-bc5f-ee599e5866e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:07.297391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93287aba-c705-4e25-bc5f-ee599e5866e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:07.297692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93287aba-c705-4e25-bc5f-ee599e5866e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:07.299047 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93287aba-c705-4e25-bc5f-ee599e5866e9 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:07.311032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 619/1243] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:07.316869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9461fda-3462-4faf-8b99-2adacf03f029 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:07.408350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:07.433802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9461fda-3462-4faf-8b99-2adacf03f029 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:07.470687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb166d-107b-4c93-bec9-481a6e50707a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:07.471340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb166d-107b-4c93-bec9-481a6e50707a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:07.471626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb166d-107b-4c93-bec9-481a6e50707a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:07.471902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb166d-107b-4c93-bec9-481a6e50707a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:07.479069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb166d-107b-4c93-bec9-481a6e50707a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:07.481356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb166d-107b-4c93-bec9-481a6e50707a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-03fb166d-107b-4c93-bec9-481a6e50707a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:22:07.493598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:07.494848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:07.496464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:22:07.496464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:22:07.496464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:22:07.496464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:22:07.496464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:22:07.496464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:22:07.496464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:22:07.496464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:07.496464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 625/1244] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:22:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:07.501722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93287aba-c705-4e25-bc5f-ee599e5866e9 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:07.616951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93287aba-c705-4e25-bc5f-ee599e5866e9 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:07.655809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874d852-f1c2-4e0b-84f9-227e45f0cdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:07.656677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874d852-f1c2-4e0b-84f9-227e45f0cdc8 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:07.656948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874d852-f1c2-4e0b-84f9-227e45f0cdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:07.657214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874d852-f1c2-4e0b-84f9-227e45f0cdc8 None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:07.658086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874d852-f1c2-4e0b-84f9-227e45f0cdc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:07.658527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874d852-f1c2-4e0b-84f9-227e45f0cdc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:07.660413 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874d852-f1c2-4e0b-84f9-227e45f0cdc8 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:07.674542 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 620/1245] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:22:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:07.680975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb166d-107b-4c93-bec9-481a6e50707a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:07.787721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb166d-107b-4c93-bec9-481a6e50707a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:07.824252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1954a-c25f-489f-bb54-1f96dd44fb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:07.824896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1954a-c25f-489f-bb54-1f96dd44fb83 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:07.825178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1954a-c25f-489f-bb54-1f96dd44fb83 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:07.825453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1954a-c25f-489f-bb54-1f96dd44fb83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:07.826179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1954a-c25f-489f-bb54-1f96dd44fb83 None admin] RBAC: Adding query filter params (role.id=3a9f17a4e0074293989007ce1a8b2565, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=1dcbe2b128a94b9abbf3a9b25db1e845) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:07.826484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1954a-c25f-489f-bb54-1f96dd44fb83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:07.827915 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1954a-c25f-489f-bb54-1f96dd44fb83 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:07.833777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 626/1246] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:22:07 2022] GET /identity/v3/role_assignments?user.id=1dcbe2b128a94b9abbf3a9b25db1e845&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=3a9f17a4e0074293989007ce1a8b2565 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:09.905928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5636627-00f5-47af-9be4-e4e32644b5fc None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:09.906482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5636627-00f5-47af-9be4-e4e32644b5fc None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:09.906482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5636627-00f5-47af-9be4-e4e32644b5fc None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:09.907808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 621/1247] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:22:09.914459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35998a-12f0-49be-bf0f-e93c7ece2942 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:09.914961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35998a-12f0-49be-bf0f-e93c7ece2942 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:09.914961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35998a-12f0-49be-bf0f-e93c7ece2942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:09.922806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:09.937405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1d35998a-12f0-49be-bf0f-e93c7ece2942 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:22:09.938834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1d35998a-12f0-49be-bf0f-e93c7ece2942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:22:10.016121 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 627/1248] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:22:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:22:10.317825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5636627-00f5-47af-9be4-e4e32644b5fc None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:10.337912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:10.441993 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5636627-00f5-47af-9be4-e4e32644b5fc None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:10.485515 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f20c6b-4b52-4cdd-afe5-8a681ca92973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:10.486187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f20c6b-4b52-4cdd-afe5-8a681ca92973 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:10.486434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f20c6b-4b52-4cdd-afe5-8a681ca92973 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:10.486768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f20c6b-4b52-4cdd-afe5-8a681ca92973 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:10.498101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f20c6b-4b52-4cdd-afe5-8a681ca92973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-f6f20c6b-4b52-4cdd-afe5-8a681ca92973 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:22:10.503155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:10.509880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:10.511651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 622/1249] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:22:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:10.513771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35998a-12f0-49be-bf0f-e93c7ece2942 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:10.627483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35998a-12f0-49be-bf0f-e93c7ece2942 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:10.665640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:10.672086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6c92c-414c-441e-b889-23774a27b3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:10.672779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6c92c-414c-441e-b889-23774a27b3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:10.673092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6c92c-414c-441e-b889-23774a27b3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:10.673387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6c92c-414c-441e-b889-23774a27b3b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:10.674129 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6c92c-414c-441e-b889-23774a27b3b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:10.674444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6c92c-414c-441e-b889-23774a27b3b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:10.675893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6c92c-414c-441e-b889-23774a27b3b9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:10.686900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 628/1250] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:22:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:10.693953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f20c6b-4b52-4cdd-afe5-8a681ca92973 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:10.782154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:10.807074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f20c6b-4b52-4cdd-afe5-8a681ca92973 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:10.845712 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad84098-8545-41aa-b1b1-3a1618e11569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:10.846350 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad84098-8545-41aa-b1b1-3a1618e11569 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:10.846687 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad84098-8545-41aa-b1b1-3a1618e11569 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:10.846953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad84098-8545-41aa-b1b1-3a1618e11569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:10.858064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad84098-8545-41aa-b1b1-3a1618e11569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:10.859701 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad84098-8545-41aa-b1b1-3a1618e11569 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-5ad84098-8545-41aa-b1b1-3a1618e11569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:22:10.877142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:10.878470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:22:10.879648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:22:10.879648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:10.879648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:10.879648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 623/1251] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:10.884411 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6c92c-414c-441e-b889-23774a27b3b9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:10.997572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6c92c-414c-441e-b889-23774a27b3b9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:11.037045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38dbcc-ab50-4022-b673-44ea20525540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:11.037673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38dbcc-ab50-4022-b673-44ea20525540 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:11.037960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38dbcc-ab50-4022-b673-44ea20525540 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:11.038278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38dbcc-ab50-4022-b673-44ea20525540 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:11.039002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38dbcc-ab50-4022-b673-44ea20525540 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:11.039312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38dbcc-ab50-4022-b673-44ea20525540 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:11.040853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38dbcc-ab50-4022-b673-44ea20525540 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:11.054875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 629/1252] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:11.061134 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad84098-8545-41aa-b1b1-3a1618e11569 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:11.181640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad84098-8545-41aa-b1b1-3a1618e11569 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:11.210637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:11.220434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3ecb9-fcfd-4190-b2aa-8c0769d7590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:11.222366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3ecb9-fcfd-4190-b2aa-8c0769d7590a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:11.222366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3ecb9-fcfd-4190-b2aa-8c0769d7590a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:11.222366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3ecb9-fcfd-4190-b2aa-8c0769d7590a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:11.230843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3ecb9-fcfd-4190-b2aa-8c0769d7590a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:11.232467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3ecb9-fcfd-4190-b2aa-8c0769d7590a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-42b3ecb9-fcfd-4190-b2aa-8c0769d7590a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:22:11.244557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:11.246284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:11.252576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:11.252576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:11.252576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:22:11.252576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:22:11.252576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:22:11.252576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:22:11.252576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:22:11.252576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:22:11.252576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 16:22:11.252576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:11.252576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 624/1253] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 16:22:11.253454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38dbcc-ab50-4022-b673-44ea20525540 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:11.318586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:11.369957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38dbcc-ab50-4022-b673-44ea20525540 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:11.409439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203100b-9212-44f9-941d-1b990a240280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:11.410188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203100b-9212-44f9-941d-1b990a240280 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:11.410455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203100b-9212-44f9-941d-1b990a240280 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:11.410738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203100b-9212-44f9-941d-1b990a240280 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:11.411619 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203100b-9212-44f9-941d-1b990a240280 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:11.411966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203100b-9212-44f9-941d-1b990a240280 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:11.413648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203100b-9212-44f9-941d-1b990a240280 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:11.436295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 630/1254] 10.222.0.55 () {56 vars in 1143 bytes} [Fri Aug 19 16:22:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:11.442400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3ecb9-fcfd-4190-b2aa-8c0769d7590a None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:11.553920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3ecb9-fcfd-4190-b2aa-8c0769d7590a None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:11.589316 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c3da7-573b-457b-bfc9-16f0bfdaabd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:11.589924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c3da7-573b-457b-bfc9-16f0bfdaabd1 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:11.590200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c3da7-573b-457b-bfc9-16f0bfdaabd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:11.590472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c3da7-573b-457b-bfc9-16f0bfdaabd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:11.650628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:11.671370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c3da7-573b-457b-bfc9-16f0bfdaabd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:11.673072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c3da7-573b-457b-bfc9-16f0bfdaabd1 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:11.792716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 625/1255] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:22:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:22:11.794019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203100b-9212-44f9-941d-1b990a240280 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:11.901587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203100b-9212-44f9-941d-1b990a240280 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:11.915216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:11.940019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327200e-7fa2-4ad2-938f-29b2ad54356e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:11.940848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327200e-7fa2-4ad2-938f-29b2ad54356e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:11.941180 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327200e-7fa2-4ad2-938f-29b2ad54356e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:11.941591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327200e-7fa2-4ad2-938f-29b2ad54356e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:11.953076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327200e-7fa2-4ad2-938f-29b2ad54356e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:11.954709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327200e-7fa2-4ad2-938f-29b2ad54356e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-7327200e-7fa2-4ad2-938f-29b2ad54356e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:22:11.973245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:11.974412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:22:11.975667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:22:11.975667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:11.975667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:11.975667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 631/1256] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:11.980766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c3da7-573b-457b-bfc9-16f0bfdaabd1 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:12.071666 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:12.085375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c3da7-573b-457b-bfc9-16f0bfdaabd1 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:12.130212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd4ac9-0e4e-43d6-8bc8-a79b2d6ccaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:12.130896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd4ac9-0e4e-43d6-8bc8-a79b2d6ccaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:12.131251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd4ac9-0e4e-43d6-8bc8-a79b2d6ccaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:12.131515 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd4ac9-0e4e-43d6-8bc8-a79b2d6ccaa6 None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:12.132379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd4ac9-0e4e-43d6-8bc8-a79b2d6ccaa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:12.132655 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd4ac9-0e4e-43d6-8bc8-a79b2d6ccaa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:12.134298 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd4ac9-0e4e-43d6-8bc8-a79b2d6ccaa6 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:12.148273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 626/1257] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:22:12.154533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327200e-7fa2-4ad2-938f-29b2ad54356e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:12.265027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327200e-7fa2-4ad2-938f-29b2ad54356e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:12.300564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de2492-044e-4389-9260-58890adc4b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:12.301311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de2492-044e-4389-9260-58890adc4b3e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:12.301594 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de2492-044e-4389-9260-58890adc4b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:12.301870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de2492-044e-4389-9260-58890adc4b3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:12.310025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de2492-044e-4389-9260-58890adc4b3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:12.311557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de2492-044e-4389-9260-58890adc4b3e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-01de2492-044e-4389-9260-58890adc4b3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:22:12.324092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:12.325357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:12.326652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:22:12.326652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:22:12.326652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:22:12.326652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:22:12.326652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:22:12.326652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:22:12.326652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:22:12.326652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:12.326652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 632/1258] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:22:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:12.333331 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd4ac9-0e4e-43d6-8bc8-a79b2d6ccaa6 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:12.441799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd4ac9-0e4e-43d6-8bc8-a79b2d6ccaa6 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:12.478121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17930a7-c25f-4a4e-b9d0-5fd8ad5ae8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:12.478721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17930a7-c25f-4a4e-b9d0-5fd8ad5ae8cc None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:12.479583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17930a7-c25f-4a4e-b9d0-5fd8ad5ae8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:12.479878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17930a7-c25f-4a4e-b9d0-5fd8ad5ae8cc None admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:12.480564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17930a7-c25f-4a4e-b9d0-5fd8ad5ae8cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:12.480896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17930a7-c25f-4a4e-b9d0-5fd8ad5ae8cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:12.482326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17930a7-c25f-4a4e-b9d0-5fd8ad5ae8cc None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:12.495259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 627/1259] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:22:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:12.501378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de2492-044e-4389-9260-58890adc4b3e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:12.532296 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:12.619040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de2492-044e-4389-9260-58890adc4b3e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:12.661022 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784260a1-f76c-43f2-bcf8-0e81dc0ca4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:12.661647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784260a1-f76c-43f2-bcf8-0e81dc0ca4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:12.661923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784260a1-f76c-43f2-bcf8-0e81dc0ca4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:12.662194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784260a1-f76c-43f2-bcf8-0e81dc0ca4d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:12.662935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784260a1-f76c-43f2-bcf8-0e81dc0ca4d6 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=1dcbe2b128a94b9abbf3a9b25db1e845) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:12.663240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784260a1-f76c-43f2-bcf8-0e81dc0ca4d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:12.664731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784260a1-f76c-43f2-bcf8-0e81dc0ca4d6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:12.671562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 633/1260] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:22:12 2022] GET /identity/v3/role_assignments?user.id=1dcbe2b128a94b9abbf3a9b25db1e845&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:14.607535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36350b4-6a37-46c9-ac8d-b9bb8a6a19cc None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:14.608085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36350b4-6a37-46c9-ac8d-b9bb8a6a19cc None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:14.608085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36350b4-6a37-46c9-ac8d-b9bb8a6a19cc None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:14.609457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 628/1261] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:22:14.614584 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bb6f0-7c86-460f-ac83-a3a5d07bec9d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:14.614994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bb6f0-7c86-460f-ac83-a3a5d07bec9d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:14.615133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bb6f0-7c86-460f-ac83-a3a5d07bec9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:14.632885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-458bb6f0-7c86-460f-ac83-a3a5d07bec9d None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:22:14.634067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-458bb6f0-7c86-460f-ac83-a3a5d07bec9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:22:14.699669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 634/1262] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:22:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:22:14.963493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36350b4-6a37-46c9-ac8d-b9bb8a6a19cc None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:14.974530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:15.062892 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36350b4-6a37-46c9-ac8d-b9bb8a6a19cc None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:15.098951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48155eec-8f67-4205-b3e5-6aa18c8ec7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:15.099749 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48155eec-8f67-4205-b3e5-6aa18c8ec7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:15.100176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48155eec-8f67-4205-b3e5-6aa18c8ec7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:15.100549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48155eec-8f67-4205-b3e5-6aa18c8ec7d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:15.113303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48155eec-8f67-4205-b3e5-6aa18c8ec7d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-48155eec-8f67-4205-b3e5-6aa18c8ec7d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:22:15.118423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:15.119441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:15.120268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 629/1263] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:22:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:15.126995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bb6f0-7c86-460f-ac83-a3a5d07bec9d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:15.145826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:15.233106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bb6f0-7c86-460f-ac83-a3a5d07bec9d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:15.267548 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd198ac-9fa8-49b3-8cb1-a420344b627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:15.268140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd198ac-9fa8-49b3-8cb1-a420344b627d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:15.268412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd198ac-9fa8-49b3-8cb1-a420344b627d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:15.268680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd198ac-9fa8-49b3-8cb1-a420344b627d None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:15.269358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd198ac-9fa8-49b3-8cb1-a420344b627d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:15.269653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd198ac-9fa8-49b3-8cb1-a420344b627d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:15.271001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd198ac-9fa8-49b3-8cb1-a420344b627d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:15.279402 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 635/1264] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:22:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:15.285281 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48155eec-8f67-4205-b3e5-6aa18c8ec7d7 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:15.352242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:15.386612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48155eec-8f67-4205-b3e5-6aa18c8ec7d7 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:15.420089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cfa94-9c0c-4db8-958c-73fd657ed241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:15.420704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cfa94-9c0c-4db8-958c-73fd657ed241 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:15.421014 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cfa94-9c0c-4db8-958c-73fd657ed241 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:15.421288 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cfa94-9c0c-4db8-958c-73fd657ed241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:15.430959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cfa94-9c0c-4db8-958c-73fd657ed241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:15.432697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cfa94-9c0c-4db8-958c-73fd657ed241 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-a30cfa94-9c0c-4db8-958c-73fd657ed241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:22:15.446716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:15.448330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:22:15.449336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:22:15.449336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:15.449336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:15.449336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 630/1265] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:15.454453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd198ac-9fa8-49b3-8cb1-a420344b627d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:15.534950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:15.554193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd198ac-9fa8-49b3-8cb1-a420344b627d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:15.586754 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded5175-c5dc-4c73-afa4-7c6a532b4e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:15.587356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded5175-c5dc-4c73-afa4-7c6a532b4e36 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:15.587638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded5175-c5dc-4c73-afa4-7c6a532b4e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:15.587914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded5175-c5dc-4c73-afa4-7c6a532b4e36 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:15.589188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded5175-c5dc-4c73-afa4-7c6a532b4e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:15.589489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded5175-c5dc-4c73-afa4-7c6a532b4e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:15.590864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded5175-c5dc-4c73-afa4-7c6a532b4e36 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:15.602521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 636/1266] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:22:15.608235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cfa94-9c0c-4db8-958c-73fd657ed241 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:15.705666 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cfa94-9c0c-4db8-958c-73fd657ed241 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:15.725016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:15.738862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b9aff-b7a6-4707-a4c6-be26cc2eee81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:15.739475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b9aff-b7a6-4707-a4c6-be26cc2eee81 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:15.739750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b9aff-b7a6-4707-a4c6-be26cc2eee81 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:15.740018 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b9aff-b7a6-4707-a4c6-be26cc2eee81 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:15.748177 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b9aff-b7a6-4707-a4c6-be26cc2eee81 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:15.749640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b9aff-b7a6-4707-a4c6-be26cc2eee81 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-df9b9aff-b7a6-4707-a4c6-be26cc2eee81 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:22:15.760727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:15.761849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:15.762965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:15.762965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:15.762965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:22:15.762965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:22:15.762965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:22:15.762965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:22:15.762965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:22:15.762965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:22:15.762965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 16:22:15.762965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:15.762965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 631/1267] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:15.767735 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded5175-c5dc-4c73-afa4-7c6a532b4e36 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:15.866912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded5175-c5dc-4c73-afa4-7c6a532b4e36 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:15.899780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4ecfd-4061-4a96-bd1e-e7d6244eec71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:15.900372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4ecfd-4061-4a96-bd1e-e7d6244eec71 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:15.900708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4ecfd-4061-4a96-bd1e-e7d6244eec71 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:15.901064 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4ecfd-4061-4a96-bd1e-e7d6244eec71 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:15.901775 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4ecfd-4061-4a96-bd1e-e7d6244eec71 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:15.902076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4ecfd-4061-4a96-bd1e-e7d6244eec71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:15.903497 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4ecfd-4061-4a96-bd1e-e7d6244eec71 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:15.911808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:15.920368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 637/1268] 10.222.0.55 () {56 vars in 1143 bytes} [Fri Aug 19 16:22:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:15.925930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b9aff-b7a6-4707-a4c6-be26cc2eee81 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:16.022277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b9aff-b7a6-4707-a4c6-be26cc2eee81 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:16.054077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6567ac-55ec-44e3-b08e-b08febb5b218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:16.054678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6567ac-55ec-44e3-b08e-b08febb5b218 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:16.054951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6567ac-55ec-44e3-b08e-b08febb5b218 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:16.055218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6567ac-55ec-44e3-b08e-b08febb5b218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:16.065967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6567ac-55ec-44e3-b08e-b08febb5b218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:16.067453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6567ac-55ec-44e3-b08e-b08febb5b218 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-af6567ac-55ec-44e3-b08e-b08febb5b218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:22:16.081518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:16.082458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:22:16.083380 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:22:16.083380 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:16.083380 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:16.083380 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 632/1269] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:16.089650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4ecfd-4061-4a96-bd1e-e7d6244eec71 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:16.194587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4ecfd-4061-4a96-bd1e-e7d6244eec71 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:16.226950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4bc23-5ef3-420b-b845-c5d8a6db71ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:16.227543 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4bc23-5ef3-420b-b845-c5d8a6db71ad None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:16.227819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4bc23-5ef3-420b-b845-c5d8a6db71ad None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:16.228087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4bc23-5ef3-420b-b845-c5d8a6db71ad None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:16.229351 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4bc23-5ef3-420b-b845-c5d8a6db71ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:16.229654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4bc23-5ef3-420b-b845-c5d8a6db71ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:16.231007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4bc23-5ef3-420b-b845-c5d8a6db71ad None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:16.241776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 638/1270] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:16.247991 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6567ac-55ec-44e3-b08e-b08febb5b218 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:16.278387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:16.343841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6567ac-55ec-44e3-b08e-b08febb5b218 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:16.376486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876a212-65bd-4ef1-b24d-f6288d92626f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:16.377114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876a212-65bd-4ef1-b24d-f6288d92626f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:16.377390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876a212-65bd-4ef1-b24d-f6288d92626f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:16.377657 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876a212-65bd-4ef1-b24d-f6288d92626f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:16.383908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876a212-65bd-4ef1-b24d-f6288d92626f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:16.385481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876a212-65bd-4ef1-b24d-f6288d92626f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-7876a212-65bd-4ef1-b24d-f6288d92626f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:22:16.396074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:16.397102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:16.398222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:22:16.398222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:22:16.398222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:22:16.398222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:22:16.398222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:22:16.398222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:22:16.398222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:22:16.398222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:16.398222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 633/1271] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:22:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:16.402931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4bc23-5ef3-420b-b845-c5d8a6db71ad None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:16.459918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:16.501426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4bc23-5ef3-420b-b845-c5d8a6db71ad None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:16.533683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edd06c-cd72-4d7a-9ca6-c60c7d2af353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:16.534278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edd06c-cd72-4d7a-9ca6-c60c7d2af353 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:16.534558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edd06c-cd72-4d7a-9ca6-c60c7d2af353 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:16.534829 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edd06c-cd72-4d7a-9ca6-c60c7d2af353 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:16.535507 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edd06c-cd72-4d7a-9ca6-c60c7d2af353 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:16.535809 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edd06c-cd72-4d7a-9ca6-c60c7d2af353 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:16.537266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edd06c-cd72-4d7a-9ca6-c60c7d2af353 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:16.549547 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 639/1272] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:22:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:16.555375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876a212-65bd-4ef1-b24d-f6288d92626f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:16.644860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:16.654508 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876a212-65bd-4ef1-b24d-f6288d92626f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:16.687546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37208984-03c0-412e-b34a-b215aa71344c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:16.688199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37208984-03c0-412e-b34a-b215aa71344c None admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:16.688481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37208984-03c0-412e-b34a-b215aa71344c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:16.688774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37208984-03c0-412e-b34a-b215aa71344c None admin] PATH_INFO: `/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/1dcbe2b128a94b9abbf3a9b25db1e845/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:16.690666 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37208984-03c0-412e-b34a-b215aa71344c None admin] RBAC: Authorizing `identity:create_grant(project_id=d143cf444f354ec0bf2089dae3d94255, user_id=1dcbe2b128a94b9abbf3a9b25db1e845, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:16.692087 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37208984-03c0-412e-b34a-b215aa71344c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:16.704308 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 634/1273] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Aug 19 16:22:16 2022] PUT /identity/v3/projects/d143cf444f354ec0bf2089dae3d94255/users/1dcbe2b128a94b9abbf3a9b25db1e845/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:22:18.578527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7e128-319e-4a14-96dd-08c137be9e29 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:18.579085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7e128-319e-4a14-96dd-08c137be9e29 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:18.579085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7e128-319e-4a14-96dd-08c137be9e29 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:18.580407 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 640/1274] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:22:18.585435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a66aa3-8c31-4d2a-9821-c88523832f27 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:18.585822 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a66aa3-8c31-4d2a-9821-c88523832f27 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:18.585989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a66aa3-8c31-4d2a-9821-c88523832f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:18.604694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-12a66aa3-8c31-4d2a-9821-c88523832f27 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:22:18.605912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-12a66aa3-8c31-4d2a-9821-c88523832f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:22:18.670522 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 635/1275] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:22:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:22:18.941241 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7e128-319e-4a14-96dd-08c137be9e29 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:19.040957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7e128-319e-4a14-96dd-08c137be9e29 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:19.058056 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:19.074981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a36a75-a7fb-449e-baf0-2590c124ede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:19.075596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a36a75-a7fb-449e-baf0-2590c124ede6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:19.075875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a36a75-a7fb-449e-baf0-2590c124ede6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:19.076147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a36a75-a7fb-449e-baf0-2590c124ede6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:19.086083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a36a75-a7fb-449e-baf0-2590c124ede6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-d9a36a75-a7fb-449e-baf0-2590c124ede6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 16:22:19.090780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:19.091816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:19.092640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 641/1276] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Aug 19 16:22:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:19.098739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a66aa3-8c31-4d2a-9821-c88523832f27 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:19.184832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:19.202985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a66aa3-8c31-4d2a-9821-c88523832f27 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:19.236561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067d3fe-bc7c-44f7-85c5-a886339ac8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:19.237184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067d3fe-bc7c-44f7-85c5-a886339ac8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:19.237463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067d3fe-bc7c-44f7-85c5-a886339ac8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:19.237732 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067d3fe-bc7c-44f7-85c5-a886339ac8b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:19.238379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067d3fe-bc7c-44f7-85c5-a886339ac8b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:19.238676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067d3fe-bc7c-44f7-85c5-a886339ac8b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:19.240035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067d3fe-bc7c-44f7-85c5-a886339ac8b2 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:19.248424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 636/1277] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Aug 19 16:22:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:19.254180 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a36a75-a7fb-449e-baf0-2590c124ede6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:19.351241 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a36a75-a7fb-449e-baf0-2590c124ede6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:19.384199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc99e1e-9c39-485f-a7ea-c0a8d7552bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:19.384880 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc99e1e-9c39-485f-a7ea-c0a8d7552bdc None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:19.385159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc99e1e-9c39-485f-a7ea-c0a8d7552bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:19.385460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc99e1e-9c39-485f-a7ea-c0a8d7552bdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:19.395614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc99e1e-9c39-485f-a7ea-c0a8d7552bdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:19.397078 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc99e1e-9c39-485f-a7ea-c0a8d7552bdc None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-2fc99e1e-9c39-485f-a7ea-c0a8d7552bdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:22:19.410627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:19.411745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:22:19.412949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:22:19.412949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:19.412949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:19.412949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 642/1278] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:19.418106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067d3fe-bc7c-44f7-85c5-a886339ac8b2 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:19.513921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067d3fe-bc7c-44f7-85c5-a886339ac8b2 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:19.547341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183b091-f7c7-4793-a633-362093ba782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:19.547940 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183b091-f7c7-4793-a633-362093ba782f None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:19.548220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183b091-f7c7-4793-a633-362093ba782f None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:19.548491 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183b091-f7c7-4793-a633-362093ba782f None admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:19.549157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183b091-f7c7-4793-a633-362093ba782f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:19.549457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183b091-f7c7-4793-a633-362093ba782f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:19.550806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183b091-f7c7-4793-a633-362093ba782f None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:19.562266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 637/1279] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:19.567764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc99e1e-9c39-485f-a7ea-c0a8d7552bdc None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:19.598186 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:19.668673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc99e1e-9c39-485f-a7ea-c0a8d7552bdc None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:19.700995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a6d89-a00a-4688-a2b4-c7d81199bde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:19.701612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a6d89-a00a-4688-a2b4-c7d81199bde4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:19.701887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a6d89-a00a-4688-a2b4-c7d81199bde4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:19.702156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a6d89-a00a-4688-a2b4-c7d81199bde4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:19.709378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a6d89-a00a-4688-a2b4-c7d81199bde4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:19.710805 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a6d89-a00a-4688-a2b4-c7d81199bde4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-f93a6d89-a00a-4688-a2b4-c7d81199bde4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 16:22:19.722294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:19.723410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:19.724366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:19.724366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:19.724366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 16:22:19.724366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 16:22:19.724366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 16:22:19.724366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 16:22:19.724366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 16:22:19.724366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 16:22:19.724366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 16:22:19.724366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:19.724366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 643/1280] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:19.729789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183b091-f7c7-4793-a633-362093ba782f None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:19.733529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:19.827544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183b091-f7c7-4793-a633-362093ba782f None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:19.861445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e939ae2-319e-4f22-90a4-dd187d9910c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:19.862038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e939ae2-319e-4f22-90a4-dd187d9910c2 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:19.862322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e939ae2-319e-4f22-90a4-dd187d9910c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:19.862597 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e939ae2-319e-4f22-90a4-dd187d9910c2 None admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:19.863316 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e939ae2-319e-4f22-90a4-dd187d9910c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:19.863621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e939ae2-319e-4f22-90a4-dd187d9910c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:19.865071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e939ae2-319e-4f22-90a4-dd187d9910c2 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:19.881664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 638/1281] 10.222.0.55 () {56 vars in 1143 bytes} [Fri Aug 19 16:22:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:19.887816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a6d89-a00a-4688-a2b4-c7d81199bde4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:19.975338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:19.985060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a6d89-a00a-4688-a2b4-c7d81199bde4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:20.017179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fe3e9-fb4c-41c6-8e5e-742010afb18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:20.017767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fe3e9-fb4c-41c6-8e5e-742010afb18d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:20.018043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fe3e9-fb4c-41c6-8e5e-742010afb18d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:20.018314 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fe3e9-fb4c-41c6-8e5e-742010afb18d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:20.082215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2fe3e9-fb4c-41c6-8e5e-742010afb18d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:20.083592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2fe3e9-fb4c-41c6-8e5e-742010afb18d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:20.175381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:20.185888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 644/1282] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Aug 19 16:22:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 16:22:20.191686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e939ae2-319e-4f22-90a4-dd187d9910c2 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:20.251747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:20.289895 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e939ae2-319e-4f22-90a4-dd187d9910c2 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:20.323009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2cb6f-328c-4ca1-83c0-08bb08064b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:20.323619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2cb6f-328c-4ca1-83c0-08bb08064b0a None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:20.323896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2cb6f-328c-4ca1-83c0-08bb08064b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:20.324165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2cb6f-328c-4ca1-83c0-08bb08064b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:20.333821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2cb6f-328c-4ca1-83c0-08bb08064b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:20.335187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2cb6f-328c-4ca1-83c0-08bb08064b0a None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-59e2cb6f-328c-4ca1-83c0-08bb08064b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 16:22:20.349307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:20.350396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 16:22:20.351704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 16:22:20.351704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 16:22:20.351704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:20.351704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 639/1283] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:22:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:20.356312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fe3e9-fb4c-41c6-8e5e-742010afb18d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:20.453936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fe3e9-fb4c-41c6-8e5e-742010afb18d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:20.487941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cdcef-2424-4a76-ae8c-a3eb5f272908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:20.488536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cdcef-2424-4a76-ae8c-a3eb5f272908 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:20.488850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cdcef-2424-4a76-ae8c-a3eb5f272908 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:20.489129 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cdcef-2424-4a76-ae8c-a3eb5f272908 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:20.489767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cdcef-2424-4a76-ae8c-a3eb5f272908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:20.490070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cdcef-2424-4a76-ae8c-a3eb5f272908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:20.491427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cdcef-2424-4a76-ae8c-a3eb5f272908 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:20.502452 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 645/1284] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Aug 19 16:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:20.507939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2cb6f-328c-4ca1-83c0-08bb08064b0a None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:20.606372 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2cb6f-328c-4ca1-83c0-08bb08064b0a None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:20.619273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:20.639306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2d3e7-64b0-487c-b738-5f98cd9580b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:20.639937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2d3e7-64b0-487c-b738-5f98cd9580b8 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:20.640219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2d3e7-64b0-487c-b738-5f98cd9580b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:20.640492 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2d3e7-64b0-487c-b738-5f98cd9580b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:20.647245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2d3e7-64b0-487c-b738-5f98cd9580b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:20.648678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2d3e7-64b0-487c-b738-5f98cd9580b8 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application [None req-b5d2d3e7-64b0-487c-b738-5f98cd9580b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 16:22:20.661130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:20.662155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:20.663230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 16:22:20.663230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 16:22:20.663230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 16:22:20.663230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 16:22:20.663230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 16:22:20.663230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 16:22:20.663230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 16:22:20.663230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: ERROR keystone.server.flask.application Aug 19 16:22:20.663230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 640/1285] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Aug 19 16:22:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:20.668005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cdcef-2424-4a76-ae8c-a3eb5f272908 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:20.716704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:20.765760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cdcef-2424-4a76-ae8c-a3eb5f272908 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:20.798580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c24c7-c102-4707-adec-c21b46442e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:20.799173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c24c7-c102-4707-adec-c21b46442e2a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:20.799448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c24c7-c102-4707-adec-c21b46442e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:20.799717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c24c7-c102-4707-adec-c21b46442e2a None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:20.800393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c24c7-c102-4707-adec-c21b46442e2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:20.800692 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c24c7-c102-4707-adec-c21b46442e2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:20.802137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c24c7-c102-4707-adec-c21b46442e2a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:20.813633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 646/1286] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Aug 19 16:22:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:20.819576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2d3e7-64b0-487c-b738-5f98cd9580b8 None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:20.915735 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2d3e7-64b0-487c-b738-5f98cd9580b8 None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:20.947621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d184a-3f52-4698-a17c-bed4ab16bd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:20.948274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d184a-3f52-4698-a17c-bed4ab16bd83 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:20.948548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d184a-3f52-4698-a17c-bed4ab16bd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:20.948839 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d184a-3f52-4698-a17c-bed4ab16bd83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:20.949536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d184a-3f52-4698-a17c-bed4ab16bd83 None admin] RBAC: Adding query filter params (role.id=6ea33d7f5f9d47509b0153ec772527df, scope.project.id=d143cf444f354ec0bf2089dae3d94255, user.id=1dcbe2b128a94b9abbf3a9b25db1e845) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:20.949836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d184a-3f52-4698-a17c-bed4ab16bd83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:20.951250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d184a-3f52-4698-a17c-bed4ab16bd83 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:20.956716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 641/1287] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Aug 19 16:22:20 2022] GET /identity/v3/role_assignments?user.id=1dcbe2b128a94b9abbf3a9b25db1e845&scope.project.id=d143cf444f354ec0bf2089dae3d94255&role.id=6ea33d7f5f9d47509b0153ec772527df => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:22.836931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f308d86-2163-4c6e-91db-5df3ffdef2f7 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:22.837508 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f308d86-2163-4c6e-91db-5df3ffdef2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:22.837607 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f308d86-2163-4c6e-91db-5df3ffdef2f7 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:22.838933 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 647/1288] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:22:22.844261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd2229-3be6-488a-9887-3ed879b3c77f None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:22.844654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd2229-3be6-488a-9887-3ed879b3c77f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:22.844828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd2229-3be6-488a-9887-3ed879b3c77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:22.862543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-7ecd2229-3be6-488a-9887-3ed879b3c77f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:22:22.863707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7ecd2229-3be6-488a-9887-3ed879b3c77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:22:22.893061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:22.930161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 642/1289] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:22:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 16:22:23.202183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f308d86-2163-4c6e-91db-5df3ffdef2f7 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:23.302525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f308d86-2163-4c6e-91db-5df3ffdef2f7 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:23.312841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:23.335113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee75c3-7e25-4c3f-afed-766603dc6ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:23.335738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee75c3-7e25-4c3f-afed-766603dc6ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:23.336019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee75c3-7e25-4c3f-afed-766603dc6ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:23.336300 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee75c3-7e25-4c3f-afed-766603dc6ef0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:23.336907 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee75c3-7e25-4c3f-afed-766603dc6ef0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:23.338890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee75c3-7e25-4c3f-afed-766603dc6ef0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-7aee75c3-7e25-4c3f-afed-766603dc6ef0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:23.347568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:23.348811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:23.349900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 648/1290] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Aug 19 16:22:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:23.355334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd2229-3be6-488a-9887-3ed879b3c77f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:23.448737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd2229-3be6-488a-9887-3ed879b3c77f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:23.481694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ba7ae-7707-4d7b-b649-b62bf9a30e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:23.482283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ba7ae-7707-4d7b-b649-b62bf9a30e5c None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:23.482556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ba7ae-7707-4d7b-b649-b62bf9a30e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:23.482824 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ba7ae-7707-4d7b-b649-b62bf9a30e5c None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:23.483434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ba7ae-7707-4d7b-b649-b62bf9a30e5c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:23.483728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ba7ae-7707-4d7b-b649-b62bf9a30e5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:23.485093 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ba7ae-7707-4d7b-b649-b62bf9a30e5c None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:23.489759 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 643/1291] 10.222.0.55 () {56 vars in 1116 bytes} [Fri Aug 19 16:22:23 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:23.495369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee75c3-7e25-4c3f-afed-766603dc6ef0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:23.590975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee75c3-7e25-4c3f-afed-766603dc6ef0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:23.623219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ecb36-8d8d-49f5-a7cc-ae2467e76293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:23.623808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ecb36-8d8d-49f5-a7cc-ae2467e76293 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:23.624088 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ecb36-8d8d-49f5-a7cc-ae2467e76293 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:23.624360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ecb36-8d8d-49f5-a7cc-ae2467e76293 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:23.624999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ecb36-8d8d-49f5-a7cc-ae2467e76293 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:23.625295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ecb36-8d8d-49f5-a7cc-ae2467e76293 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:23.626647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ecb36-8d8d-49f5-a7cc-ae2467e76293 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:23.630910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 649/1292] 10.222.0.55 () {56 vars in 1116 bytes} [Fri Aug 19 16:22:23 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:25.486769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95a86-0ee5-465e-87ea-f34a13227c4f None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:25.487322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95a86-0ee5-465e-87ea-f34a13227c4f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:25.487322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95a86-0ee5-465e-87ea-f34a13227c4f None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:25.488622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 644/1293] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:22:25.493697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369650ce-67d9-430f-a2e8-4f8bd15eb866 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:25.494087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369650ce-67d9-430f-a2e8-4f8bd15eb866 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:25.494346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369650ce-67d9-430f-a2e8-4f8bd15eb866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:25.512374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-369650ce-67d9-430f-a2e8-4f8bd15eb866 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:22:25.513589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-369650ce-67d9-430f-a2e8-4f8bd15eb866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:22:25.543210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:25.580570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 650/1294] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:22:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:22:25.847721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a95a86-0ee5-465e-87ea-f34a13227c4f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:25.877953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:25.945851 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a95a86-0ee5-465e-87ea-f34a13227c4f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:25.977632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604765e-359f-43f3-829c-f4df539f72b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:25.978754 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604765e-359f-43f3-829c-f4df539f72b6 None admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:25.979031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604765e-359f-43f3-829c-f4df539f72b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:25.979299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604765e-359f-43f3-829c-f4df539f72b6 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:25.979938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604765e-359f-43f3-829c-f4df539f72b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:25.981325 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604765e-359f-43f3-829c-f4df539f72b6 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:25.995392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 645/1295] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Aug 19 16:22:25 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:22:27.869388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51a890-420d-4a9e-86de-9b7d96850653 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:27.869944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51a890-420d-4a9e-86de-9b7d96850653 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:27.869944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51a890-420d-4a9e-86de-9b7d96850653 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:27.871264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 651/1296] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:22:27.876381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36232-1e37-4d58-bd27-55a19dc1c073 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:27.876808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36232-1e37-4d58-bd27-55a19dc1c073 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:27.876967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36232-1e37-4d58-bd27-55a19dc1c073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:27.894457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-c9e36232-1e37-4d58-bd27-55a19dc1c073 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:22:27.895632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c9e36232-1e37-4d58-bd27-55a19dc1c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:22:27.965593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 646/1297] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:22:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:22:28.235566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51a890-420d-4a9e-86de-9b7d96850653 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:28.336211 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51a890-420d-4a9e-86de-9b7d96850653 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:28.369177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443ca42-1e4c-4fc0-9f79-ee9464b83243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:28.369795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443ca42-1e4c-4fc0-9f79-ee9464b83243 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:28.370070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443ca42-1e4c-4fc0-9f79-ee9464b83243 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:28.370340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443ca42-1e4c-4fc0-9f79-ee9464b83243 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:28.370905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443ca42-1e4c-4fc0-9f79-ee9464b83243 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:28.372255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443ca42-1e4c-4fc0-9f79-ee9464b83243 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-6443ca42-1e4c-4fc0-9f79-ee9464b83243 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:28.380631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:28.381768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:28.383048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 652/1298] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Aug 19 16:22:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:28.388222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36232-1e37-4d58-bd27-55a19dc1c073 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:28.406703 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:28.485385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36232-1e37-4d58-bd27-55a19dc1c073 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:28.518236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bf68a-0609-4b2b-b5e0-e4fc367d13af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:28.518764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bf68a-0609-4b2b-b5e0-e4fc367d13af None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:28.519035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bf68a-0609-4b2b-b5e0-e4fc367d13af None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:28.519302 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bf68a-0609-4b2b-b5e0-e4fc367d13af None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:28.519910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bf68a-0609-4b2b-b5e0-e4fc367d13af None admin] RBAC: Adding query filter params (name=placement) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:28.520204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bf68a-0609-4b2b-b5e0-e4fc367d13af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:28.521577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bf68a-0609-4b2b-b5e0-e4fc367d13af None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:28.527071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 647/1299] 10.222.0.55 () {56 vars in 1116 bytes} [Fri Aug 19 16:22:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:28.532998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443ca42-1e4c-4fc0-9f79-ee9464b83243 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:28.564680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:28.631147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443ca42-1e4c-4fc0-9f79-ee9464b83243 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:28.663989 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb92ad-d9de-4fc9-9411-5b996fbe4551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:28.664595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb92ad-d9de-4fc9-9411-5b996fbe4551 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:28.664911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb92ad-d9de-4fc9-9411-5b996fbe4551 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:28.665284 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb92ad-d9de-4fc9-9411-5b996fbe4551 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:28.666078 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb92ad-d9de-4fc9-9411-5b996fbe4551 None admin] RBAC: Adding query filter params (interface=public, service_id=8f1ea6faa39244bb803e24fffcce73e3, region_id=RegionOne) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:28.666383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb92ad-d9de-4fc9-9411-5b996fbe4551 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:28.667739 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb92ad-d9de-4fc9-9411-5b996fbe4551 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:28.672624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 653/1300] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Aug 19 16:22:28 2022] GET /identity/v3/endpoints?service_id=8f1ea6faa39244bb803e24fffcce73e3&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:28.678213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bf68a-0609-4b2b-b5e0-e4fc367d13af None admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:28.756564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:28.775436 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bf68a-0609-4b2b-b5e0-e4fc367d13af None admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:28.808277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e0a7c-6ee7-4703-8cbc-7d912bc263f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:28.808899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e0a7c-6ee7-4703-8cbc-7d912bc263f8 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:28.809179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e0a7c-6ee7-4703-8cbc-7d912bc263f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:28.809449 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e0a7c-6ee7-4703-8cbc-7d912bc263f8 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:28.810043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e0a7c-6ee7-4703-8cbc-7d912bc263f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:28.811392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e0a7c-6ee7-4703-8cbc-7d912bc263f8 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:28.817956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 648/1301] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Aug 19 16:22:28 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:22:30.680061 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8c864-b98b-42e3-87e1-d322346569bb None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:30.680556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8c864-b98b-42e3-87e1-d322346569bb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:30.681352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8c864-b98b-42e3-87e1-d322346569bb None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:30.682615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 654/1302] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:22:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:22:30.687591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a301950-7fc1-406b-a581-c9ca66921e1b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:30.688351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a301950-7fc1-406b-a581-c9ca66921e1b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:30.688623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a301950-7fc1-406b-a581-c9ca66921e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:30.706071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-2a301950-7fc1-406b-a581-c9ca66921e1b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:22:30.707247 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2a301950-7fc1-406b-a581-c9ca66921e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:22:30.771447 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 649/1303] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:22:30 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 16:22:31.042329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8c864-b98b-42e3-87e1-d322346569bb None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:31.135662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:31.142568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8c864-b98b-42e3-87e1-d322346569bb None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:31.175109 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1686464-0f34-47ae-8384-a771d32529af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:31.175729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1686464-0f34-47ae-8384-a771d32529af None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:31.176009 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1686464-0f34-47ae-8384-a771d32529af None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:31.176281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1686464-0f34-47ae-8384-a771d32529af None admin] PATH_INFO: `/v3/services/placement` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:31.176887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1686464-0f34-47ae-8384-a771d32529af None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:31.178250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1686464-0f34-47ae-8384-a771d32529af None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a1686464-0f34-47ae-8384-a771d32529af None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 16:22:31.186648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 16:22:31.187783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Aug 19 16:22:31.188729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 655/1304] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Aug 19 16:22:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 16:22:31.193672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a301950-7fc1-406b-a581-c9ca66921e1b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:31.269912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:31.292620 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a301950-7fc1-406b-a581-c9ca66921e1b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:31.324834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637e4bb-cfb2-4f26-b0dd-8edc5438a281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:31.325430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637e4bb-cfb2-4f26-b0dd-8edc5438a281 None admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:31.325706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637e4bb-cfb2-4f26-b0dd-8edc5438a281 None admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:31.325973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637e4bb-cfb2-4f26-b0dd-8edc5438a281 None admin] PATH_INFO: `/v3/services` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:31.326587 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637e4bb-cfb2-4f26-b0dd-8edc5438a281 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:22:31.326882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637e4bb-cfb2-4f26-b0dd-8edc5438a281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:31.328234 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637e4bb-cfb2-4f26-b0dd-8edc5438a281 None admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:31.333180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 650/1305] 10.222.0.55 () {56 vars in 1116 bytes} [Fri Aug 19 16:22:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:22:31.339611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1686464-0f34-47ae-8384-a771d32529af None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:22:31.435055 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1686464-0f34-47ae-8384-a771d32529af None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:22:31.467750 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf8672-2d6c-4a84-b90b-aa3ef01f5137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:22:31.468600 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf8672-2d6c-4a84-b90b-aa3ef01f5137 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:22:31.468901 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf8672-2d6c-4a84-b90b-aa3ef01f5137 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:22:31.469169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf8672-2d6c-4a84-b90b-aa3ef01f5137 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:22:31.469816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf8672-2d6c-4a84-b90b-aa3ef01f5137 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:22:31.471164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf8672-2d6c-4a84-b90b-aa3ef01f5137 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:22:31.489584 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 656/1306] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:22:31 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:22:41.330295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=30 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:22:41.485386 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:34.971103 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbe3b2-9b7c-416c-b5cb-5a0754a2abba None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:34.972677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbe3b2-9b7c-416c-b5cb-5a0754a2abba None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:34.973219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbe3b2-9b7c-416c-b5cb-5a0754a2abba None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:34.978858 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 651/1307] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:23:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:23:34.980186 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcfee7-e04d-4ad9-b1e9-90d46ce7a1fe None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:34.980601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcfee7-e04d-4ad9-b1e9-90d46ce7a1fe None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:34.980843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcfee7-e04d-4ad9-b1e9-90d46ce7a1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:35.107308 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1afcfee7-e04d-4ad9-b1e9-90d46ce7a1fe None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:23:35.109223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1afcfee7-e04d-4ad9-b1e9-90d46ce7a1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:35.330444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 657/1308] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:23:35.626573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500e699-ea95-4a90-b5f7-9ea2bf8e9405 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:35.626972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500e699-ea95-4a90-b5f7-9ea2bf8e9405 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:35.627315 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500e699-ea95-4a90-b5f7-9ea2bf8e9405 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:35.629643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 652/1309] 10.222.0.55 () {54 vars in 965 bytes} [Fri Aug 19 16:23:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:23:35.635780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d10c-edb4-4ad5-a79c-cbd5d382c765 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:35.636078 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d10c-edb4-4ad5-a79c-cbd5d382c765 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:35.636423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d10c-edb4-4ad5-a79c-cbd5d382c765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:35.728688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ee90d10c-edb4-4ad5-a79c-cbd5d382c765 None None] MFA Rules not processed for user `9e0b7efc488a42ffab8ba7e2d101ced4`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:23:35.730065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ee90d10c-edb4-4ad5-a79c-cbd5d382c765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:35.755293 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:35.821772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 658/1310] 10.222.0.55 () {58 vars in 1062 bytes} [Fri Aug 19 16:23:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:23:35.831290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500e699-ea95-4a90-b5f7-9ea2bf8e9405 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:23:35.833637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a500e699-ea95-4a90-b5f7-9ea2bf8e9405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:35.958432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500e699-ea95-4a90-b5f7-9ea2bf8e9405 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:23:36.037059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13f82b-dc8f-4ddf-9cb6-3762df67eb58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:23:36.037726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13f82b-dc8f-4ddf-9cb6-3762df67eb58 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:36.037998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13f82b-dc8f-4ddf-9cb6-3762df67eb58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:36.038202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13f82b-dc8f-4ddf-9cb6-3762df67eb58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:36.039823 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5a13f82b-dc8f-4ddf-9cb6-3762df67eb58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:36.098314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:36.168216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13f82b-dc8f-4ddf-9cb6-3762df67eb58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:23:36.169575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13f82b-dc8f-4ddf-9cb6-3762df67eb58 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:23:36.189567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 653/1311] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:23:38.424664 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130a56f-b980-4e12-8da9-e8a0ae2ef902 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:38.425212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130a56f-b980-4e12-8da9-e8a0ae2ef902 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:38.425316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130a56f-b980-4e12-8da9-e8a0ae2ef902 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:38.426934 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 659/1312] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:23:38.432711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cbfeb-f33e-48b8-8cb8-5e4a68d36bd4 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:38.433245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cbfeb-f33e-48b8-8cb8-5e4a68d36bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:38.433245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cbfeb-f33e-48b8-8cb8-5e4a68d36bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:38.529178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-465cbfeb-f33e-48b8-8cb8-5e4a68d36bd4 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:23:38.530530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-465cbfeb-f33e-48b8-8cb8-5e4a68d36bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:38.632295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 654/1313] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:23:38.915513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130a56f-b980-4e12-8da9-e8a0ae2ef902 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:23:38.917113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d130a56f-b980-4e12-8da9-e8a0ae2ef902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:39.022625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130a56f-b980-4e12-8da9-e8a0ae2ef902 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:23:39.094828 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:39.097316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed0c56-8e71-4414-a612-33ae84aef97a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:23:39.097921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed0c56-8e71-4414-a612-33ae84aef97a service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:39.098442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed0c56-8e71-4414-a612-33ae84aef97a service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:39.098884 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed0c56-8e71-4414-a612-33ae84aef97a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:39.100683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-59ed0c56-8e71-4414-a612-33ae84aef97a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:39.216431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed0c56-8e71-4414-a612-33ae84aef97a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:23:39.217839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed0c56-8e71-4414-a612-33ae84aef97a service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:23:39.238101 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 660/1314] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:23:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:23:41.415774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bd611-beaf-4114-a06f-f2a77a398e95 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:41.416349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bd611-beaf-4114-a06f-f2a77a398e95 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:41.416457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bd611-beaf-4114-a06f-f2a77a398e95 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:41.417793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 655/1315] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:23:41.422836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a7941-78fb-438a-ad40-44d48911f856 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:41.423244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a7941-78fb-438a-ad40-44d48911f856 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:41.423393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a7941-78fb-438a-ad40-44d48911f856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:41.487423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-603a7941-78fb-438a-ad40-44d48911f856 None None] MFA Rules not processed for user `17297aae8d6f4c7b801260350bd22b41`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:23:41.488663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-603a7941-78fb-438a-ad40-44d48911f856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:41.539760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:41.584504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 661/1316] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:23:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:23:41.868877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bd611-beaf-4114-a06f-f2a77a398e95 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:23:41.870446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9e5bd611-beaf-4114-a06f-f2a77a398e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:41.880558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:41.973256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bd611-beaf-4114-a06f-f2a77a398e95 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:23:42.044394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4dd4e-b3a3-4ef3-9065-3e8e8bfc2be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:23:42.045021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4dd4e-b3a3-4ef3-9065-3e8e8bfc2be7 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:42.045328 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4dd4e-b3a3-4ef3-9065-3e8e8bfc2be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:42.045599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4dd4e-b3a3-4ef3-9065-3e8e8bfc2be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:42.047230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b8f4dd4e-b3a3-4ef3-9065-3e8e8bfc2be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:42.110628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:42.157226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4dd4e-b3a3-4ef3-9065-3e8e8bfc2be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:23:42.158647 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4dd4e-b3a3-4ef3-9065-3e8e8bfc2be7 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:23:42.176830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 656/1317] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:23:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:23:44.620659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb32979-0b4d-41a8-bec8-1734a692d4e4 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:44.621228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb32979-0b4d-41a8-bec8-1734a692d4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:44.621228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb32979-0b4d-41a8-bec8-1734a692d4e4 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:44.622585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 662/1318] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:23:44.628447 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee940a8-48bb-4685-8bfe-73c2a6e05dcb None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:44.628898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee940a8-48bb-4685-8bfe-73c2a6e05dcb None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:44.629034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee940a8-48bb-4685-8bfe-73c2a6e05dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:44.694054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3ee940a8-48bb-4685-8bfe-73c2a6e05dcb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:23:44.695266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3ee940a8-48bb-4685-8bfe-73c2a6e05dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:44.788918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 657/1319] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:23:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:23:45.076293 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb32979-0b4d-41a8-bec8-1734a692d4e4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:23:45.077886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8bb32979-0b4d-41a8-bec8-1734a692d4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:45.185106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb32979-0b4d-41a8-bec8-1734a692d4e4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:23:45.258655 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9953f-850b-4219-a725-d680ded72a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:23:45.259259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9953f-850b-4219-a725-d680ded72a74 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:45.259543 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9953f-850b-4219-a725-d680ded72a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:45.259821 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9953f-850b-4219-a725-d680ded72a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:45.261461 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-64b9953f-850b-4219-a725-d680ded72a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:45.269360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:45.370135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9953f-850b-4219-a725-d680ded72a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:23:45.371506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9953f-850b-4219-a725-d680ded72a74 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:23:45.391454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 663/1320] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:23:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:23:47.928594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52e630-7217-4d3d-ad49-0b52c4f53112 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:47.929124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52e630-7217-4d3d-ad49-0b52c4f53112 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:47.929124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52e630-7217-4d3d-ad49-0b52c4f53112 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:47.930414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 658/1321] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:23:47.935504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc66525-ae77-4216-ab5d-141b9c782207 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:47.935944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc66525-ae77-4216-ab5d-141b9c782207 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:47.936069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc66525-ae77-4216-ab5d-141b9c782207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:48.001883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5cc66525-ae77-4216-ab5d-141b9c782207 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:23:48.003606 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5cc66525-ae77-4216-ab5d-141b9c782207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:48.059434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:48.099328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 664/1322] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:23:48.379695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52e630-7217-4d3d-ad49-0b52c4f53112 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:23:48.381369 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-cd52e630-7217-4d3d-ad49-0b52c4f53112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:48.400870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:48.486075 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52e630-7217-4d3d-ad49-0b52c4f53112 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:23:48.562195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19366631-507a-4089-9edb-b397330bddc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:23:48.562806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19366631-507a-4089-9edb-b397330bddc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:48.563077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19366631-507a-4089-9edb-b397330bddc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:48.563341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19366631-507a-4089-9edb-b397330bddc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:48.565005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-19366631-507a-4089-9edb-b397330bddc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:48.637397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:48.681295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19366631-507a-4089-9edb-b397330bddc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:23:48.682742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19366631-507a-4089-9edb-b397330bddc8 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:23:48.702115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 659/1323] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:23:50.674102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1db5d-a03a-4509-a6fe-cb1daf57b7c7 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:50.674756 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1db5d-a03a-4509-a6fe-cb1daf57b7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:50.674756 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1db5d-a03a-4509-a6fe-cb1daf57b7c7 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:50.676160 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 665/1324] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:23:50.681320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4efaa-7598-4c2f-9752-15ef68a4800b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:50.681736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4efaa-7598-4c2f-9752-15ef68a4800b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:50.681870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4efaa-7598-4c2f-9752-15ef68a4800b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:50.746704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f7b4efaa-7598-4c2f-9752-15ef68a4800b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:23:50.747911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f7b4efaa-7598-4c2f-9752-15ef68a4800b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:50.845396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 660/1325] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:23:51.152422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8aafa-f0b0-48a5-8580-a6b5701760c6 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:51.152717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8aafa-f0b0-48a5-8580-a6b5701760c6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:51.153025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8aafa-f0b0-48a5-8580-a6b5701760c6 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:51.154373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 666/1326] 10.222.0.55 () {54 vars in 965 bytes} [Fri Aug 19 16:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:23:51.160598 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526852f-0745-4f78-ba7f-331dbd83b9de None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:51.160982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526852f-0745-4f78-ba7f-331dbd83b9de None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:51.161330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526852f-0745-4f78-ba7f-331dbd83b9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:51.193237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:51.238471 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-5526852f-0745-4f78-ba7f-331dbd83b9de None None] MFA Rules not processed for user `9e0b7efc488a42ffab8ba7e2d101ced4`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:23:51.239746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5526852f-0745-4f78-ba7f-331dbd83b9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:51.324870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 661/1327] 10.222.0.55 () {58 vars in 1062 bytes} [Fri Aug 19 16:23:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:23:51.334170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8aafa-f0b0-48a5-8580-a6b5701760c6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:23:51.335717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fcb8aafa-f0b0-48a5-8580-a6b5701760c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:51.443536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8aafa-f0b0-48a5-8580-a6b5701760c6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:23:51.518460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3684cc-f3dd-4498-9b56-803622cae148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:23:51.519001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3684cc-f3dd-4498-9b56-803622cae148 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:51.519283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3684cc-f3dd-4498-9b56-803622cae148 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:51.519558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3684cc-f3dd-4498-9b56-803622cae148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:51.521183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6a3684cc-f3dd-4498-9b56-803622cae148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:51.538180 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:51.633313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3684cc-f3dd-4498-9b56-803622cae148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:23:51.634778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3684cc-f3dd-4498-9b56-803622cae148 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:23:51.656324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 667/1328] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:23:53.839611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b9860-81c6-49b2-9138-049d49ce4070 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:53.840143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b9860-81c6-49b2-9138-049d49ce4070 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:53.840143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b9860-81c6-49b2-9138-049d49ce4070 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:53.842464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 662/1329] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:23:53.846886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887ba9b-1591-4b6a-ac97-acb330a1fd93 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:53.847307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887ba9b-1591-4b6a-ac97-acb330a1fd93 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:53.847415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887ba9b-1591-4b6a-ac97-acb330a1fd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:53.931014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b887ba9b-1591-4b6a-ac97-acb330a1fd93 None None] MFA Rules not processed for user `17297aae8d6f4c7b801260350bd22b41`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:23:53.932263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b887ba9b-1591-4b6a-ac97-acb330a1fd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:54.011460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:54.052247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 668/1330] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:23:53 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:23:54.386446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b9860-81c6-49b2-9138-049d49ce4070 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:23:54.387997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5f8b9860-81c6-49b2-9138-049d49ce4070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:54.506243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b9860-81c6-49b2-9138-049d49ce4070 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:23:54.545632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:54.624553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1414bc-7bb4-45b8-a6ac-c0f20a0535d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:23:54.625231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1414bc-7bb4-45b8-a6ac-c0f20a0535d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:54.625506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1414bc-7bb4-45b8-a6ac-c0f20a0535d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:54.625773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1414bc-7bb4-45b8-a6ac-c0f20a0535d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:54.627817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9d1414bc-7bb4-45b8-a6ac-c0f20a0535d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:54.772905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1414bc-7bb4-45b8-a6ac-c0f20a0535d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:23:54.774429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1414bc-7bb4-45b8-a6ac-c0f20a0535d4 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:23:54.795790 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 663/1331] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:23:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:23:57.101506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7de72c-237b-4539-a2ee-42e04d70c55d None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:57.102060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7de72c-237b-4539-a2ee-42e04d70c55d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:57.102060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7de72c-237b-4539-a2ee-42e04d70c55d None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:57.103378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 669/1332] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:23:57.108387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20153154-b5d0-422c-b31c-0c93c48390a7 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:57.108810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20153154-b5d0-422c-b31c-0c93c48390a7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:57.108963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20153154-b5d0-422c-b31c-0c93c48390a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:57.170401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:57.173079 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-20153154-b5d0-422c-b31c-0c93c48390a7 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:23:57.175495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-20153154-b5d0-422c-b31c-0c93c48390a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:57.270157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 664/1333] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:23:57.549717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7de72c-237b-4539-a2ee-42e04d70c55d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:23:57.551249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1c7de72c-237b-4539-a2ee-42e04d70c55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:57.653349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7de72c-237b-4539-a2ee-42e04d70c55d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:23:57.724650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1ba78-bf14-4ec8-99c0-32287e10f68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:23:57.725272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1ba78-bf14-4ec8-99c0-32287e10f68c service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:57.725549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1ba78-bf14-4ec8-99c0-32287e10f68c service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:57.725818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1ba78-bf14-4ec8-99c0-32287e10f68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:57.727398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7ba1ba78-bf14-4ec8-99c0-32287e10f68c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:57.753822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:57.833780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1ba78-bf14-4ec8-99c0-32287e10f68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:23:57.835129 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1ba78-bf14-4ec8-99c0-32287e10f68c service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:23:57.852865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 670/1334] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:23:59.787991 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73737178-9195-4663-ae3b-06b20402ddb7 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:59.788533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73737178-9195-4663-ae3b-06b20402ddb7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:59.788533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73737178-9195-4663-ae3b-06b20402ddb7 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:59.789919 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 665/1335] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:23:59.795087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16627571-bd1d-45c9-9bdb-aea90d5d5324 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:23:59.795497 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16627571-bd1d-45c9-9bdb-aea90d5d5324 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:23:59.795672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16627571-bd1d-45c9-9bdb-aea90d5d5324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:23:59.861988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-16627571-bd1d-45c9-9bdb-aea90d5d5324 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:23:59.863204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-16627571-bd1d-45c9-9bdb-aea90d5d5324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:23:59.935697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:23:59.959418 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 671/1336] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:00.247127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73737178-9195-4663-ae3b-06b20402ddb7 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:00.248670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-73737178-9195-4663-ae3b-06b20402ddb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:00.354364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73737178-9195-4663-ae3b-06b20402ddb7 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:00.382401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:00.426646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5a605-2722-456a-93ce-d646b8594bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:00.427250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a605-2722-456a-93ce-d646b8594bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:00.427528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a605-2722-456a-93ce-d646b8594bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:00.427798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a605-2722-456a-93ce-d646b8594bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:00.429411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e7b5a605-2722-456a-93ce-d646b8594bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:00.537015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5a605-2722-456a-93ce-d646b8594bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:00.538440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5a605-2722-456a-93ce-d646b8594bc8 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:00.556355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 666/1337] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:02.785649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330ca2b-3051-4b1c-af58-dc7d9cfb73a5 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:02.786164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330ca2b-3051-4b1c-af58-dc7d9cfb73a5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:02.786164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330ca2b-3051-4b1c-af58-dc7d9cfb73a5 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:02.787457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 672/1338] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:02.792630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0986f36-9a6f-4b9f-8a49-2abe270d0eaf None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:02.793053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0986f36-9a6f-4b9f-8a49-2abe270d0eaf None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:02.793209 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0986f36-9a6f-4b9f-8a49-2abe270d0eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:02.856761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b0986f36-9a6f-4b9f-8a49-2abe270d0eaf None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:02.858003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b0986f36-9a6f-4b9f-8a49-2abe270d0eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:02.866627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:02.951601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 667/1339] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:03.233698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330ca2b-3051-4b1c-af58-dc7d9cfb73a5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:03.235258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6330ca2b-3051-4b1c-af58-dc7d9cfb73a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:03.338528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330ca2b-3051-4b1c-af58-dc7d9cfb73a5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:03.413587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a96b2-5d19-4aab-8033-545154998217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:03.414194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a96b2-5d19-4aab-8033-545154998217 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:03.414474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a96b2-5d19-4aab-8033-545154998217 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:03.414747 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a96b2-5d19-4aab-8033-545154998217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:03.416335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-115a96b2-5d19-4aab-8033-545154998217 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:03.450649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:03.525562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a96b2-5d19-4aab-8033-545154998217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:03.527005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a96b2-5d19-4aab-8033-545154998217 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:03.544625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 673/1340] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:07.392568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45cbb-d01d-4446-bcbf-8962f2559278 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:07.393152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45cbb-d01d-4446-bcbf-8962f2559278 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:07.393152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45cbb-d01d-4446-bcbf-8962f2559278 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:07.394492 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 668/1341] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:07.399706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a473a-8fdf-4f78-a806-6674fbac9500 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:07.400092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a473a-8fdf-4f78-a806-6674fbac9500 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:07.400269 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a473a-8fdf-4f78-a806-6674fbac9500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:07.466682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e96a473a-8fdf-4f78-a806-6674fbac9500 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:07.467891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e96a473a-8fdf-4f78-a806-6674fbac9500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:07.555293 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:07.568791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 674/1342] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:07.851970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c45cbb-d01d-4446-bcbf-8962f2559278 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:07.853647 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e2c45cbb-d01d-4446-bcbf-8962f2559278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:07.968653 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c45cbb-d01d-4446-bcbf-8962f2559278 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:08.005582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:08.041262 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57aa5e-b25e-434f-8245-ac4aaf2c2392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:08.041869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57aa5e-b25e-434f-8245-ac4aaf2c2392 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:08.042146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57aa5e-b25e-434f-8245-ac4aaf2c2392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:08.042414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57aa5e-b25e-434f-8245-ac4aaf2c2392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:08.043992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-cf57aa5e-b25e-434f-8245-ac4aaf2c2392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:08.152914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf57aa5e-b25e-434f-8245-ac4aaf2c2392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:08.154348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf57aa5e-b25e-434f-8245-ac4aaf2c2392 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:08.171841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 669/1343] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:10.737411 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a4357-a76b-4f84-aeef-6cc502776a17 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:10.737954 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a4357-a76b-4f84-aeef-6cc502776a17 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:10.737954 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a4357-a76b-4f84-aeef-6cc502776a17 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:10.739289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 675/1344] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:10.744528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b9d2c-7340-41b0-b64d-b0016596f384 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:10.744988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b9d2c-7340-41b0-b64d-b0016596f384 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:10.745091 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b9d2c-7340-41b0-b64d-b0016596f384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:10.813257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-fa0b9d2c-7340-41b0-b64d-b0016596f384 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:10.814396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-fa0b9d2c-7340-41b0-b64d-b0016596f384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:10.832939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:10.913192 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 670/1345] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:24:11.192659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a4357-a76b-4f84-aeef-6cc502776a17 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:11.194277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-066a4357-a76b-4f84-aeef-6cc502776a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:11.300316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a4357-a76b-4f84-aeef-6cc502776a17 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:11.378772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55985139-109f-4bfe-a0a6-d27b4f943494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:11.379375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55985139-109f-4bfe-a0a6-d27b4f943494 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:11.379651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55985139-109f-4bfe-a0a6-d27b4f943494 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:11.379921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55985139-109f-4bfe-a0a6-d27b4f943494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:11.381545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-55985139-109f-4bfe-a0a6-d27b4f943494 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:11.424599 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:11.490664 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55985139-109f-4bfe-a0a6-d27b4f943494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:11.492114 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55985139-109f-4bfe-a0a6-d27b4f943494 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:11.511236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 676/1346] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:15.419229 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c184fa-9141-44e6-b9d8-9cd124bac693 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:15.419702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c184fa-9141-44e6-b9d8-9cd124bac693 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:15.419804 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c184fa-9141-44e6-b9d8-9cd124bac693 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:15.421205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 671/1347] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:15.427097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154b7ec-f217-4db4-8d0f-2e9666ffd2de None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:15.427615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154b7ec-f217-4db4-8d0f-2e9666ffd2de None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:15.427718 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154b7ec-f217-4db4-8d0f-2e9666ffd2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:15.499002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f154b7ec-f217-4db4-8d0f-2e9666ffd2de None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:15.500784 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f154b7ec-f217-4db4-8d0f-2e9666ffd2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:15.598978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:15.602070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 677/1348] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:15.884582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c184fa-9141-44e6-b9d8-9cd124bac693 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:15.886226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-09c184fa-9141-44e6-b9d8-9cd124bac693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:15.995373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c184fa-9141-44e6-b9d8-9cd124bac693 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:16.043331 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:16.069679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccea75-0286-44a5-9ebb-185e06b93289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:16.070289 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccea75-0286-44a5-9ebb-185e06b93289 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:16.070563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccea75-0286-44a5-9ebb-185e06b93289 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:16.070832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccea75-0286-44a5-9ebb-185e06b93289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:16.072404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8bccea75-0286-44a5-9ebb-185e06b93289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:16.179518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bccea75-0286-44a5-9ebb-185e06b93289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:16.180904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bccea75-0286-44a5-9ebb-185e06b93289 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:16.198311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 672/1349] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:21.062578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e3759-8183-47ea-9dfd-bf0161b09aab None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:21.063523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e3759-8183-47ea-9dfd-bf0161b09aab None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:21.064004 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e3759-8183-47ea-9dfd-bf0161b09aab None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:21.066376 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 678/1350] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:24:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:21.073029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfa0d6-44de-4671-aa16-cfd8ef1fd7c2 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:21.073456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfa0d6-44de-4671-aa16-cfd8ef1fd7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:21.073637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfa0d6-44de-4671-aa16-cfd8ef1fd7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:21.165360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-6bdfa0d6-44de-4671-aa16-cfd8ef1fd7c2 None None] MFA Rules not processed for user `1dcbe2b128a94b9abbf3a9b25db1e845`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:21.166840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6bdfa0d6-44de-4671-aa16-cfd8ef1fd7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:21.186895 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:21.311501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 673/1351] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:24:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:25.556843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8db414-9ab2-4fc0-b905-d5c7257994a4 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:25.557433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8db414-9ab2-4fc0-b905-d5c7257994a4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:25.557433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8db414-9ab2-4fc0-b905-d5c7257994a4 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:25.558890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 679/1352] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:25.563746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6a00-bf5d-4b1b-aea1-4f33d01b4087 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:25.564212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6a00-bf5d-4b1b-aea1-4f33d01b4087 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:25.564544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6a00-bf5d-4b1b-aea1-4f33d01b4087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:25.648691 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-320b6a00-bf5d-4b1b-aea1-4f33d01b4087 None None] MFA Rules not processed for user `1dcbe2b128a94b9abbf3a9b25db1e845`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:25.650022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-320b6a00-bf5d-4b1b-aea1-4f33d01b4087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:25.729667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:25.769197 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 674/1353] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:24:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:27.159371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bf81a-904d-4e9f-9c68-ed269e38d5f4 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:27.159923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bf81a-904d-4e9f-9c68-ed269e38d5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:27.159923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bf81a-904d-4e9f-9c68-ed269e38d5f4 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:27.161283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 680/1354] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:27.166300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b04da5-2d0c-43b2-b766-5cb857b5389d None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:27.167042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b04da5-2d0c-43b2-b766-5cb857b5389d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:27.167320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b04da5-2d0c-43b2-b766-5cb857b5389d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:27.242415 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-20b04da5-2d0c-43b2-b766-5cb857b5389d None None] MFA Rules not processed for user `1dcbe2b128a94b9abbf3a9b25db1e845`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:27.243616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-20b04da5-2d0c-43b2-b766-5cb857b5389d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:27.351669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 675/1355] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:24:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:30.618884 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29b5e6-435c-4624-9b4c-96aaf9d4c4e1 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:30.619429 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29b5e6-435c-4624-9b4c-96aaf9d4c4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:30.619614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29b5e6-435c-4624-9b4c-96aaf9d4c4e1 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:30.620980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 681/1356] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:30.626187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead95180-ffaf-4fc6-b6c1-14f37dbee736 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:30.626594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead95180-ffaf-4fc6-b6c1-14f37dbee736 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:30.626742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead95180-ffaf-4fc6-b6c1-14f37dbee736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:30.671179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:30.710972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ead95180-ffaf-4fc6-b6c1-14f37dbee736 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:30.712154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ead95180-ffaf-4fc6-b6c1-14f37dbee736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:30.854261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 676/1357] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:31.151272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed82580-622b-408c-b161-4b24489d5ec5 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:31.151769 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed82580-622b-408c-b161-4b24489d5ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:31.152080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed82580-622b-408c-b161-4b24489d5ec5 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:31.153546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 682/1358] 10.222.0.55 () {54 vars in 962 bytes} [Fri Aug 19 16:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:31.161405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950e330-930c-4405-aba5-828f3b86b051 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:31.161972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950e330-930c-4405-aba5-828f3b86b051 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:31.162324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950e330-930c-4405-aba5-828f3b86b051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:31.247384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-c950e330-930c-4405-aba5-828f3b86b051 None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:31.249386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c950e330-930c-4405-aba5-828f3b86b051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:31.299457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:31.379431 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 677/1359] 10.222.0.55 () {58 vars in 1059 bytes} [Fri Aug 19 16:24:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:24:31.387348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed82580-622b-408c-b161-4b24489d5ec5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:31.389688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6ed82580-622b-408c-b161-4b24489d5ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:31.549217 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed82580-622b-408c-b161-4b24489d5ec5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:31.627714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:31.670913 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915097b-0896-48bf-8a39-3fb64a5de9d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:31.671561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915097b-0896-48bf-8a39-3fb64a5de9d4 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:31.671914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915097b-0896-48bf-8a39-3fb64a5de9d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:31.672249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915097b-0896-48bf-8a39-3fb64a5de9d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:31.673931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f915097b-0896-48bf-8a39-3fb64a5de9d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:31.767974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915097b-0896-48bf-8a39-3fb64a5de9d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:31.769610 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915097b-0896-48bf-8a39-3fb64a5de9d4 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:31.789409 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 683/1360] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:24:33.903578 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfae259-7058-489e-9644-9a3f5cbc4853 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:33.904053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfae259-7058-489e-9644-9a3f5cbc4853 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:33.904053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfae259-7058-489e-9644-9a3f5cbc4853 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:33.905619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 678/1361] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:33.910703 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c4baa-ea62-4e04-ba59-4fb482346430 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:33.911155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c4baa-ea62-4e04-ba59-4fb482346430 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:33.911266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c4baa-ea62-4e04-ba59-4fb482346430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:33.981332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3a2c4baa-ea62-4e04-ba59-4fb482346430 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:33.983079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3a2c4baa-ea62-4e04-ba59-4fb482346430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:34.023261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:34.087941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 684/1362] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:34.384221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfae259-7058-489e-9644-9a3f5cbc4853 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:34.385903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9cfae259-7058-489e-9644-9a3f5cbc4853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:34.558245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfae259-7058-489e-9644-9a3f5cbc4853 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:34.568200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:34.692947 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202264aa-a25a-43b5-b15c-242545ab3afe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:34.693625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202264aa-a25a-43b5-b15c-242545ab3afe service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:34.693972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202264aa-a25a-43b5-b15c-242545ab3afe service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:34.694308 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202264aa-a25a-43b5-b15c-242545ab3afe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:34.696115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-202264aa-a25a-43b5-b15c-242545ab3afe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:34.801604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202264aa-a25a-43b5-b15c-242545ab3afe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:34.803219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202264aa-a25a-43b5-b15c-242545ab3afe service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:34.822382 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 679/1363] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:37.057575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d8e3e-4b31-45bc-a426-d78d37be8148 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:37.058075 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d8e3e-4b31-45bc-a426-d78d37be8148 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:37.058075 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d8e3e-4b31-45bc-a426-d78d37be8148 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:37.059361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 685/1364] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:37.065656 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc61e31-0a6d-4347-be27-5459a50c649a None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:37.066105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc61e31-0a6d-4347-be27-5459a50c649a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:37.066922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc61e31-0a6d-4347-be27-5459a50c649a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:37.084639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:37.148670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-1fc61e31-0a6d-4347-be27-5459a50c649a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:37.149933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-1fc61e31-0a6d-4347-be27-5459a50c649a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:37.253811 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 680/1365] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:37.540823 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dfb41-5d78-4475-a310-8eaa755bbcdc None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:37.541322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dfb41-5d78-4475-a310-8eaa755bbcdc None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:37.541601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dfb41-5d78-4475-a310-8eaa755bbcdc None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:37.542921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 686/1366] 10.222.0.55 () {54 vars in 962 bytes} [Fri Aug 19 16:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:37.549311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb231b-7a64-46b5-a495-8735bfe35320 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:37.549665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb231b-7a64-46b5-a495-8735bfe35320 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:37.550011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb231b-7a64-46b5-a495-8735bfe35320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:37.629730 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-c5fb231b-7a64-46b5-a495-8735bfe35320 None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:37.631035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c5fb231b-7a64-46b5-a495-8735bfe35320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:37.678404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:37.765428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 681/1367] 10.222.0.55 () {58 vars in 1059 bytes} [Fri Aug 19 16:24:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:37.773200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dfb41-5d78-4475-a310-8eaa755bbcdc None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:37.774723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a67dfb41-5d78-4475-a310-8eaa755bbcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:37.930185 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dfb41-5d78-4475-a310-8eaa755bbcdc None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:37.954183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:38.055230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17820a38-a5d0-4a72-9487-be4afd60dc8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:38.055836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17820a38-a5d0-4a72-9487-be4afd60dc8a service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:38.056113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17820a38-a5d0-4a72-9487-be4afd60dc8a service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:38.056383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17820a38-a5d0-4a72-9487-be4afd60dc8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:38.057993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-17820a38-a5d0-4a72-9487-be4afd60dc8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:38.149937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17820a38-a5d0-4a72-9487-be4afd60dc8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:38.151575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17820a38-a5d0-4a72-9487-be4afd60dc8a service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:38.172829 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 687/1368] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:38.776048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71bbcc-9937-4b85-828b-ca339c631d48 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:38.776622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71bbcc-9937-4b85-828b-ca339c631d48 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:38.776914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71bbcc-9937-4b85-828b-ca339c631d48 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:38.778424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 682/1369] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:38.784147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8770d44-54d8-4890-86ee-c1c7e7a3a626 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:38.784437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8770d44-54d8-4890-86ee-c1c7e7a3a626 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:38.784712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8770d44-54d8-4890-86ee-c1c7e7a3a626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:38.819851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:38.859185 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e8770d44-54d8-4890-86ee-c1c7e7a3a626 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:38.860403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e8770d44-54d8-4890-86ee-c1c7e7a3a626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:38.964797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 688/1370] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:39.545263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e202a-30da-4554-9260-cb7e55e3238f None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:39.545263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e202a-30da-4554-9260-cb7e55e3238f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:39.545263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e202a-30da-4554-9260-cb7e55e3238f None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:39.547255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 683/1371] 10.222.0.55 () {54 vars in 964 bytes} [Fri Aug 19 16:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:39.550722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd46a2-227d-4e71-baba-a9759b02ed49 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:39.551209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd46a2-227d-4e71-baba-a9759b02ed49 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:39.551363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd46a2-227d-4e71-baba-a9759b02ed49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:39.631143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3cbd46a2-227d-4e71-baba-a9759b02ed49 None None] MFA Rules not processed for user `1eaa3980ad8840879e4e426b00e3c56f`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:39.632366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3cbd46a2-227d-4e71-baba-a9759b02ed49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:39.680955 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:39.715871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 689/1372] 10.222.0.55 () {58 vars in 1061 bytes} [Fri Aug 19 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:39.720828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e202a-30da-4554-9260-cb7e55e3238f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:39.722331 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a17e202a-30da-4554-9260-cb7e55e3238f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:39.831132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e202a-30da-4554-9260-cb7e55e3238f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:39.878034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:39.921611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad5bf2-39c5-44bd-9a23-32514e57c3aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:39.922232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad5bf2-39c5-44bd-9a23-32514e57c3aa service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:39.922536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad5bf2-39c5-44bd-9a23-32514e57c3aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:39.922853 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad5bf2-39c5-44bd-9a23-32514e57c3aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:39.924384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-96ad5bf2-39c5-44bd-9a23-32514e57c3aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:40.061272 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad5bf2-39c5-44bd-9a23-32514e57c3aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:40.062544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad5bf2-39c5-44bd-9a23-32514e57c3aa service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:40.085195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 684/1373] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:40.151549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2138d-e1f8-44e7-9695-c657122bc99f None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:40.151549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2138d-e1f8-44e7-9695-c657122bc99f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:40.151549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2138d-e1f8-44e7-9695-c657122bc99f None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:40.151549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 690/1374] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:40.156476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ca26f-dbd9-44a5-9363-908536b6f6f7 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:40.156792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ca26f-dbd9-44a5-9363-908536b6f6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:40.157074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ca26f-dbd9-44a5-9363-908536b6f6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:40.228614 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ff6ca26f-dbd9-44a5-9363-908536b6f6f7 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:40.229876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ff6ca26f-dbd9-44a5-9363-908536b6f6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:40.253925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:40.335346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 685/1375] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:40.636245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2138d-e1f8-44e7-9695-c657122bc99f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:40.637827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-68c2138d-e1f8-44e7-9695-c657122bc99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:40.791473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2138d-e1f8-44e7-9695-c657122bc99f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:40.846131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:40.913705 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51eaa1e-f35d-424d-9f52-317ac7bd2f55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:40.914215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51eaa1e-f35d-424d-9f52-317ac7bd2f55 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:40.914494 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51eaa1e-f35d-424d-9f52-317ac7bd2f55 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:40.914764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51eaa1e-f35d-424d-9f52-317ac7bd2f55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:40.916366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b51eaa1e-f35d-424d-9f52-317ac7bd2f55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:41.005535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51eaa1e-f35d-424d-9f52-317ac7bd2f55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:41.007140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51eaa1e-f35d-424d-9f52-317ac7bd2f55 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:41.026239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 691/1376] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:42.977777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503be16c-a33d-4ad3-a0d4-696c37a8879d None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:42.978304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503be16c-a33d-4ad3-a0d4-696c37a8879d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:42.978403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503be16c-a33d-4ad3-a0d4-696c37a8879d None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:42.979761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 686/1377] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:42.985591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f454f-244e-42a7-9272-dcd228cb396f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:42.986007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f454f-244e-42a7-9272-dcd228cb396f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:42.986163 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f454f-244e-42a7-9272-dcd228cb396f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:43.055164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bd4f454f-244e-42a7-9272-dcd228cb396f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:43.056385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bd4f454f-244e-42a7-9272-dcd228cb396f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:43.063702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:43.180930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 692/1378] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:43.478322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503be16c-a33d-4ad3-a0d4-696c37a8879d None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:43.479919 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-503be16c-a33d-4ad3-a0d4-696c37a8879d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:43.681464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503be16c-a33d-4ad3-a0d4-696c37a8879d None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:43.686646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:43.807745 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b6bab-3225-4f68-84c8-4eb86491c6b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:43.808352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b6bab-3225-4f68-84c8-4eb86491c6b6 service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:43.808634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b6bab-3225-4f68-84c8-4eb86491c6b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:43.808936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b6bab-3225-4f68-84c8-4eb86491c6b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:43.810569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3b2b6bab-3225-4f68-84c8-4eb86491c6b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:43.908620 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b6bab-3225-4f68-84c8-4eb86491c6b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:43.910242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b6bab-3225-4f68-84c8-4eb86491c6b6 service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:43.929679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 687/1379] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:45.866567 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf8a5a-da9a-4d4c-920c-dfc8094ad38e None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:45.867114 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf8a5a-da9a-4d4c-920c-dfc8094ad38e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:45.867114 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf8a5a-da9a-4d4c-920c-dfc8094ad38e None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:45.868589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 693/1380] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:45.874617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46522a49-60e4-459e-bfba-fd0a7b87a0a7 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:45.874998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46522a49-60e4-459e-bfba-fd0a7b87a0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:45.875377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46522a49-60e4-459e-bfba-fd0a7b87a0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:45.889941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:45.957267 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-46522a49-60e4-459e-bfba-fd0a7b87a0a7 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:45.958851 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-46522a49-60e4-459e-bfba-fd0a7b87a0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:46.068795 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 688/1381] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:46.356501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf8a5a-da9a-4d4c-920c-dfc8094ad38e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:46.358639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-50cf8a5a-da9a-4d4c-920c-dfc8094ad38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:46.530257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:46.541896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf8a5a-da9a-4d4c-920c-dfc8094ad38e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:46.682538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610834ca-42bc-42ea-b656-37d8ab143420 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:46.683144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610834ca-42bc-42ea-b656-37d8ab143420 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:46.683419 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610834ca-42bc-42ea-b656-37d8ab143420 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:46.683688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610834ca-42bc-42ea-b656-37d8ab143420 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:46.686275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-610834ca-42bc-42ea-b656-37d8ab143420 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:46.779303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610834ca-42bc-42ea-b656-37d8ab143420 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:46.780922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610834ca-42bc-42ea-b656-37d8ab143420 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:46.791577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:46.804903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 694/1382] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:24:48.794644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd28c0-3c1e-4fe8-9380-c803a6075296 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:48.795304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd28c0-3c1e-4fe8-9380-c803a6075296 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:48.795304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd28c0-3c1e-4fe8-9380-c803a6075296 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:48.796575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 689/1383] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:48.802625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf9b0a-b11b-44a2-a14c-0af9b7026c3e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:48.803075 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf9b0a-b11b-44a2-a14c-0af9b7026c3e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:48.803261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf9b0a-b11b-44a2-a14c-0af9b7026c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:48.886271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a9bf9b0a-b11b-44a2-a14c-0af9b7026c3e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:48.888468 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a9bf9b0a-b11b-44a2-a14c-0af9b7026c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:48.999540 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 695/1384] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:49.294768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd28c0-3c1e-4fe8-9380-c803a6075296 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:49.296734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-37cd28c0-3c1e-4fe8-9380-c803a6075296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:49.391632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:49.463644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd28c0-3c1e-4fe8-9380-c803a6075296 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:49.585164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccba6d8-fb5a-41d0-bedc-47b8cf9792ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:49.585700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccba6d8-fb5a-41d0-bedc-47b8cf9792ea service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:49.585975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccba6d8-fb5a-41d0-bedc-47b8cf9792ea service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:49.586242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccba6d8-fb5a-41d0-bedc-47b8cf9792ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:49.587881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-4ccba6d8-fb5a-41d0-bedc-47b8cf9792ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:49.640848 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:49.686679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccba6d8-fb5a-41d0-bedc-47b8cf9792ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:49.688407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccba6d8-fb5a-41d0-bedc-47b8cf9792ea service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:49.707352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 690/1385] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:51.678511 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7a68a-58f6-4021-8498-a0ff19a4b15b None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:51.679038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7a68a-58f6-4021-8498-a0ff19a4b15b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:51.679204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7a68a-58f6-4021-8498-a0ff19a4b15b None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:51.681135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 696/1386] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:51.686950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82af8-79ba-4eb5-978d-987646b33091 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:51.687443 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82af8-79ba-4eb5-978d-987646b33091 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:51.687443 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82af8-79ba-4eb5-978d-987646b33091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:51.756371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-55b82af8-79ba-4eb5-978d-987646b33091 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:51.757668 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-55b82af8-79ba-4eb5-978d-987646b33091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:51.867151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 691/1387] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:52.167809 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7a68a-58f6-4021-8498-a0ff19a4b15b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:52.170295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-60d7a68a-58f6-4021-8498-a0ff19a4b15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:52.255739 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:52.332337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7a68a-58f6-4021-8498-a0ff19a4b15b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:52.447661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99d41f-4183-456b-bc54-21efc9d3cbbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:52.448208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99d41f-4183-456b-bc54-21efc9d3cbbe service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:52.448488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99d41f-4183-456b-bc54-21efc9d3cbbe service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:52.448783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99d41f-4183-456b-bc54-21efc9d3cbbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:52.450572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-db99d41f-4183-456b-bc54-21efc9d3cbbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:52.491639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:52.545861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99d41f-4183-456b-bc54-21efc9d3cbbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:52.547468 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99d41f-4183-456b-bc54-21efc9d3cbbe service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:52.565793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 697/1388] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:54.528952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4861c4-27b0-40ee-ba51-e786af0fc79e None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:54.529529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4861c4-27b0-40ee-ba51-e786af0fc79e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:54.529529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4861c4-27b0-40ee-ba51-e786af0fc79e None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:54.530858 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 692/1389] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:54.536105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33f0c9-8676-483f-a048-29dcda73fc88 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:54.536525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33f0c9-8676-483f-a048-29dcda73fc88 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:54.536642 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33f0c9-8676-483f-a048-29dcda73fc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:54.606832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4a33f0c9-8676-483f-a048-29dcda73fc88 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:54.608070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4a33f0c9-8676-483f-a048-29dcda73fc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:54.707529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 698/1390] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:24:55.006215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4861c4-27b0-40ee-ba51-e786af0fc79e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:55.007772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ff4861c4-27b0-40ee-ba51-e786af0fc79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:55.041211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:55.168654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4861c4-27b0-40ee-ba51-e786af0fc79e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:55.278986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d7358-d828-406d-82a7-ed490f3880f0 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:55.279178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d7358-d828-406d-82a7-ed490f3880f0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:55.279446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d7358-d828-406d-82a7-ed490f3880f0 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:55.280841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 699/1391] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:24:55.286552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403f631-717a-4316-93c5-344877b29d3e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:55.286847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:55.286981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403f631-717a-4316-93c5-344877b29d3e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:55.287448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403f631-717a-4316-93c5-344877b29d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:55.301881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947c2c3-b4e8-474c-9565-8d80f23cad7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:55.302488 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947c2c3-b4e8-474c-9565-8d80f23cad7c service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:55.302792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947c2c3-b4e8-474c-9565-8d80f23cad7c service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:55.303062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947c2c3-b4e8-474c-9565-8d80f23cad7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:55.304702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-1947c2c3-b4e8-474c-9565-8d80f23cad7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:55.307213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:55.377614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c403f631-717a-4316-93c5-344877b29d3e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:55.381113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c403f631-717a-4316-93c5-344877b29d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:55.417403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947c2c3-b4e8-474c-9565-8d80f23cad7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:55.419240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947c2c3-b4e8-474c-9565-8d80f23cad7c service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:55.439887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 693/1392] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:55.519113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 700/1393] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:55.867837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb57d-d79e-4546-a2ad-5421c31bc7e1 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:55.868414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb57d-d79e-4546-a2ad-5421c31bc7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:55.868414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb57d-d79e-4546-a2ad-5421c31bc7e1 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:55.869754 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 694/1394] 10.222.0.55 () {54 vars in 967 bytes} [Fri Aug 19 16:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:24:55.875493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327b64c-2c74-4b38-af74-00378538c95c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:55.875912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327b64c-2c74-4b38-af74-00378538c95c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:55.876041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327b64c-2c74-4b38-af74-00378538c95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:55.956091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6327b64c-2c74-4b38-af74-00378538c95c None None] MFA Rules not processed for user `748ad823b944441d907994cda6d8ac7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:55.957260 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6327b64c-2c74-4b38-af74-00378538c95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:55.993758 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:56.037408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 701/1395] 10.222.0.55 () {58 vars in 1064 bytes} [Fri Aug 19 16:24:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:56.045646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fb57d-d79e-4546-a2ad-5421c31bc7e1 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:56.047163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a70fb57d-d79e-4546-a2ad-5421c31bc7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:56.150076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fb57d-d79e-4546-a2ad-5421c31bc7e1 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:56.163957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:56.224788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9923fe-f053-4676-b832-2d3ac68a18ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '748ad823b944441d907994cda6d8ac7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:56.225348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9923fe-f053-4676-b832-2d3ac68a18ac service glance] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:56.225623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9923fe-f053-4676-b832-2d3ac68a18ac service glance] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:56.225893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9923fe-f053-4676-b832-2d3ac68a18ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:56.227495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-dd9923fe-f053-4676-b832-2d3ac68a18ac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:56.342516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9923fe-f053-4676-b832-2d3ac68a18ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:56.343980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9923fe-f053-4676-b832-2d3ac68a18ac service glance] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:56.362254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 695/1396] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:24:57.105178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afe69c-af03-4861-ad7a-a8d3830f4816 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:57.106232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afe69c-af03-4861-ad7a-a8d3830f4816 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:57.106643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afe69c-af03-4861-ad7a-a8d3830f4816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:57.174839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-23afe69c-af03-4861-ad7a-a8d3830f4816 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:57.176065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-23afe69c-af03-4861-ad7a-a8d3830f4816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:57.264911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 702/1397] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:24:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:24:57.312590 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b9669-34be-4a0c-b9b6-a793d9c024a2 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:57.313128 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b9669-34be-4a0c-b9b6-a793d9c024a2 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:57.313234 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b9669-34be-4a0c-b9b6-a793d9c024a2 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:57.314554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 696/1398] 10.222.0.55 () {54 vars in 943 bytes} [Fri Aug 19 16:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:24:57.321497 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f717b-6a0d-4e7c-864a-d441781c6027 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:57.321803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f717b-6a0d-4e7c-864a-d441781c6027 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:57.322062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f717b-6a0d-4e7c-864a-d441781c6027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:57.379956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:57.400174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ff5f717b-6a0d-4e7c-864a-d441781c6027 None None] MFA Rules not processed for user `c755f9538e984bc3877112896595e158`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:57.401413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ff5f717b-6a0d-4e7c-864a-d441781c6027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:57.419897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9784a1-429c-43bb-9580-e4e99521127b None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:57.420178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9784a1-429c-43bb-9580-e4e99521127b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:57.420458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9784a1-429c-43bb-9580-e4e99521127b None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:57.421847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 697/1399] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:24:57.426854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4fbc5-c141-4790-9c41-d423e0b8fef6 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:57.427138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4fbc5-c141-4790-9c41-d423e0b8fef6 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:57.427414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4fbc5-c141-4790-9c41-d423e0b8fef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:57.482492 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:57.497452 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-54c4fbc5-c141-4790-9c41-d423e0b8fef6 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:24:57.498677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-54c4fbc5-c141-4790-9c41-d423e0b8fef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:57.512330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 703/1400] 10.222.0.55 () {58 vars in 1040 bytes} [Fri Aug 19 16:24:57 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:24:57.521865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f717b-6a0d-4e7c-864a-d441781c6027 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:57.534397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f717b-6a0d-4e7c-864a-d441781c6027 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:57.598446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 698/1401] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:24:57.634377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:57.643943 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd9bbc-a35a-4507-835f-6b0114302029 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:57.644560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd9bbc-a35a-4507-835f-6b0114302029 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:57.644864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd9bbc-a35a-4507-835f-6b0114302029 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:57.645145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd9bbc-a35a-4507-835f-6b0114302029 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:57.646806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e3dd9bbc-a35a-4507-835f-6b0114302029 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:57.731403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd9bbc-a35a-4507-835f-6b0114302029 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:57.733154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd9bbc-a35a-4507-835f-6b0114302029 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:57.741409 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 704/1402] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:24:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 19 16:24:57.895916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4fbc5-c141-4790-9c41-d423e0b8fef6 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:24:57.897189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-54c4fbc5-c141-4790-9c41-d423e0b8fef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:58.018730 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4fbc5-c141-4790-9c41-d423e0b8fef6 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:24:58.034852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:24:58.150622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22693e-4fe9-4192-9441-6bfa78522b4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:24:58.151262 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22693e-4fe9-4192-9441-6bfa78522b4b service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:24:58.151604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22693e-4fe9-4192-9441-6bfa78522b4b service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:24:58.151949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22693e-4fe9-4192-9441-6bfa78522b4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:24:58.153995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ea22693e-4fe9-4192-9441-6bfa78522b4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:24:58.256321 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22693e-4fe9-4192-9441-6bfa78522b4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:24:58.257965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22693e-4fe9-4192-9441-6bfa78522b4b service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:24:58.280621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 699/1403] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:00.212580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542672ae-9947-41f3-9fe6-2f37bab17264 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:00.213164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542672ae-9947-41f3-9fe6-2f37bab17264 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:00.213357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542672ae-9947-41f3-9fe6-2f37bab17264 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:00.215651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 705/1404] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:25:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:00.220622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4d072-5d0e-4c92-beaa-f0189db00e07 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:00.221097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4d072-5d0e-4c92-beaa-f0189db00e07 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:00.221191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4d072-5d0e-4c92-beaa-f0189db00e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:00.243947 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:00.297651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-c1a4d072-5d0e-4c92-beaa-f0189db00e07 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:00.299162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c1a4d072-5d0e-4c92-beaa-f0189db00e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:00.418472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 700/1405] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:00.715448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542672ae-9947-41f3-9fe6-2f37bab17264 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:00.717012 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-542672ae-9947-41f3-9fe6-2f37bab17264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:00.901280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:00.916303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542672ae-9947-41f3-9fe6-2f37bab17264 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:01.050069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566450f6-52e5-498c-a235-254c1f335db5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:01.050700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566450f6-52e5-498c-a235-254c1f335db5 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:01.050984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566450f6-52e5-498c-a235-254c1f335db5 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:01.051286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566450f6-52e5-498c-a235-254c1f335db5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:01.053016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-566450f6-52e5-498c-a235-254c1f335db5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:01.153837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566450f6-52e5-498c-a235-254c1f335db5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:01.157656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566450f6-52e5-498c-a235-254c1f335db5 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:01.167291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:01.177126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 706/1406] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:03.141576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bec766-e1ea-409c-9354-133ecb37c14d None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:03.142143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bec766-e1ea-409c-9354-133ecb37c14d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:03.142143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bec766-e1ea-409c-9354-133ecb37c14d None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:03.144828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 701/1407] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:03.149111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6dfdb-88eb-42d6-abac-c41e3e507a38 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:03.149538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6dfdb-88eb-42d6-abac-c41e3e507a38 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:03.149979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6dfdb-88eb-42d6-abac-c41e3e507a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:03.225462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c6e6dfdb-88eb-42d6-abac-c41e3e507a38 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:03.227080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c6e6dfdb-88eb-42d6-abac-c41e3e507a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:03.335786 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 707/1408] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:03.628861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bec766-e1ea-409c-9354-133ecb37c14d None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:03.630480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-04bec766-e1ea-409c-9354-133ecb37c14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:03.746313 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:03.813434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bec766-e1ea-409c-9354-133ecb37c14d None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:03.941725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ec309-7581-4b31-a4db-98fd132f24c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:03.942930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ec309-7581-4b31-a4db-98fd132f24c7 service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:03.943285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ec309-7581-4b31-a4db-98fd132f24c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:03.943627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ec309-7581-4b31-a4db-98fd132f24c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:03.945346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-110ec309-7581-4b31-a4db-98fd132f24c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:04.014979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:04.056375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ec309-7581-4b31-a4db-98fd132f24c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:04.059454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ec309-7581-4b31-a4db-98fd132f24c7 service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:04.078614 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 702/1409] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:13.333971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=67 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:14.076297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=20 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:19.721204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3b505-825e-4c21-a082-5565eeceb7d0 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:19.721752 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3b505-825e-4c21-a082-5565eeceb7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:19.721752 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3b505-825e-4c21-a082-5565eeceb7d0 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:19.723098 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 708/1410] 10.222.0.55 () {54 vars in 918 bytes} [Fri Aug 19 16:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:19.728637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0b0d8-7c4a-4978-9fda-3c50cce5eb17 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:19.729093 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0b0d8-7c4a-4978-9fda-3c50cce5eb17 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:19.729199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0b0d8-7c4a-4978-9fda-3c50cce5eb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:19.803200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-84f0b0d8-7c4a-4978-9fda-3c50cce5eb17 None None] MFA Rules not processed for user `1dcbe2b128a94b9abbf3a9b25db1e845`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:19.804415 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-84f0b0d8-7c4a-4978-9fda-3c50cce5eb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:19.910103 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 703/1411] 10.222.0.55 () {58 vars in 1015 bytes} [Fri Aug 19 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:19.924224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b99308-e875-4e97-862e-51749e85f22f None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:19.924587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b99308-e875-4e97-862e-51749e85f22f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:19.924921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b99308-e875-4e97-862e-51749e85f22f None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:19.926288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 709/1412] 10.222.0.55 () {54 vars in 918 bytes} [Fri Aug 19 16:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:19.930733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6cc3d-d4ee-47ac-bd92-b5163fe41872 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:19.931020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6cc3d-d4ee-47ac-bd92-b5163fe41872 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:19.931292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6cc3d-d4ee-47ac-bd92-b5163fe41872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:20.003501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:20.006629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-71f6cc3d-d4ee-47ac-bd92-b5163fe41872 None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:20.008439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-71f6cc3d-d4ee-47ac-bd92-b5163fe41872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:20.122138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 704/1413] 10.222.0.55 () {58 vars in 1015 bytes} [Fri Aug 19 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:25:22.485953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39543f51-1fba-4e16-8521-c1b81dd2361a None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:22.486500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39543f51-1fba-4e16-8521-c1b81dd2361a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:22.486500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39543f51-1fba-4e16-8521-c1b81dd2361a None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:22.487811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:22.492959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0e84f-b79e-4bf2-81db-38107e03f63e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:22.493345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0e84f-b79e-4bf2-81db-38107e03f63e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:22.493546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0e84f-b79e-4bf2-81db-38107e03f63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:22.559183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-98c0e84f-b79e-4bf2-81db-38107e03f63e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:22.560438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-98c0e84f-b79e-4bf2-81db-38107e03f63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:22.615305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:22.657939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:22.945719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39543f51-1fba-4e16-8521-c1b81dd2361a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:22.947243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-39543f51-1fba-4e16-8521-c1b81dd2361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:23.091776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39543f51-1fba-4e16-8521-c1b81dd2361a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:23.163664 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:23.205573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58078b-f8ef-4b55-b40e-57e7f68c0b4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:23.206119 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58078b-f8ef-4b55-b40e-57e7f68c0b4d service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:23.206403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58078b-f8ef-4b55-b40e-57e7f68c0b4d service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:23.206680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58078b-f8ef-4b55-b40e-57e7f68c0b4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:23.208557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ae58078b-f8ef-4b55-b40e-57e7f68c0b4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:23.297993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58078b-f8ef-4b55-b40e-57e7f68c0b4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:23.299579 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58078b-f8ef-4b55-b40e-57e7f68c0b4d service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:23.318094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 711/1416] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:24.199966 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4186fc0-4902-4da7-9daf-4171a47b18e7 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:24.200540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4186fc0-4902-4da7-9daf-4171a47b18e7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:24.200540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4186fc0-4902-4da7-9daf-4171a47b18e7 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:24.201924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 16:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:24.206908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a0ec9-620f-4585-ab15-ca6a8981e4f3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:24.207319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a0ec9-620f-4585-ab15-ca6a8981e4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:24.207464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a0ec9-620f-4585-ab15-ca6a8981e4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:24.283563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bf6a0ec9-620f-4585-ab15-ca6a8981e4f3 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:24.284798 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bf6a0ec9-620f-4585-ab15-ca6a8981e4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:24.311332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:24.381110 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 16:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:24.529486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4186fc0-4902-4da7-9daf-4171a47b18e7 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:24.531002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e4186fc0-4902-4da7-9daf-4171a47b18e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:24.675724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4186fc0-4902-4da7-9daf-4171a47b18e7 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:24.715843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:24.788459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9714e-e899-4a1c-94bc-8e1a323902f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:24.789127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9714e-e899-4a1c-94bc-8e1a323902f6 service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:24.789437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9714e-e899-4a1c-94bc-8e1a323902f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:24.789736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9714e-e899-4a1c-94bc-8e1a323902f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:24.791351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8fe9714e-e899-4a1c-94bc-8e1a323902f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:24.881851 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9714e-e899-4a1c-94bc-8e1a323902f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:24.883450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9714e-e899-4a1c-94bc-8e1a323902f6 service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:24.901260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 707/1419] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:25.784858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431d3df-25e8-4441-b7a2-0fb8ba79373b None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:25.785384 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431d3df-25e8-4441-b7a2-0fb8ba79373b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:25.785384 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431d3df-25e8-4441-b7a2-0fb8ba79373b None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:25.786752 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 16:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:25.791601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715aa47e-1489-4aab-9ef0-580bd87c5b69 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:25.792033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715aa47e-1489-4aab-9ef0-580bd87c5b69 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:25.792160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715aa47e-1489-4aab-9ef0-580bd87c5b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:25.848190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:25.871180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-715aa47e-1489-4aab-9ef0-580bd87c5b69 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:25.872744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-715aa47e-1489-4aab-9ef0-580bd87c5b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:25.969758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 16:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:26.125724 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431d3df-25e8-4441-b7a2-0fb8ba79373b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:26.127402 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d431d3df-25e8-4441-b7a2-0fb8ba79373b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:26.273895 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431d3df-25e8-4441-b7a2-0fb8ba79373b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:26.286829 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:26.385608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e5ded-7ff6-44d6-90bc-640d93caedd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:26.386720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e5ded-7ff6-44d6-90bc-640d93caedd6 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:26.387035 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e5ded-7ff6-44d6-90bc-640d93caedd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:26.387304 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e5ded-7ff6-44d6-90bc-640d93caedd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:26.388937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-139e5ded-7ff6-44d6-90bc-640d93caedd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:26.478714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e5ded-7ff6-44d6-90bc-640d93caedd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:26.480469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e5ded-7ff6-44d6-90bc-640d93caedd6 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:26.498181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 714/1422] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:27.370724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b415d1-6bef-440d-a83c-f093eabd2bb3 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:27.371869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b415d1-6bef-440d-a83c-f093eabd2bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:27.371869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b415d1-6bef-440d-a83c-f093eabd2bb3 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:27.373226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 16:25:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:27.378381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790196b0-7675-484a-a486-6dc0f6ca41d1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:27.378799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790196b0-7675-484a-a486-6dc0f6ca41d1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:27.378941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790196b0-7675-484a-a486-6dc0f6ca41d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:27.398485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:27.454610 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-790196b0-7675-484a-a486-6dc0f6ca41d1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:27.455815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-790196b0-7675-484a-a486-6dc0f6ca41d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:27.551604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 16:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:27.702197 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b415d1-6bef-440d-a83c-f093eabd2bb3 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:27.703717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-24b415d1-6bef-440d-a83c-f093eabd2bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:27.812500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:27.850117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b415d1-6bef-440d-a83c-f093eabd2bb3 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:27.962409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fa588-f310-457f-b13c-339e9e1bf93d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:27.962935 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fa588-f310-457f-b13c-339e9e1bf93d service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:27.963217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fa588-f310-457f-b13c-339e9e1bf93d service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:27.963494 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fa588-f310-457f-b13c-339e9e1bf93d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:27.965788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2f5fa588-f310-457f-b13c-339e9e1bf93d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:28.040190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:28.054206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fa588-f310-457f-b13c-339e9e1bf93d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:28.055799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fa588-f310-457f-b13c-339e9e1bf93d service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:28.074530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 710/1425] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:28.944291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b76fa1-174b-40b4-89d5-ba1181815933 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:28.944876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b76fa1-174b-40b4-89d5-ba1181815933 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:28.944876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b76fa1-174b-40b4-89d5-ba1181815933 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:28.946149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 16:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:28.951105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc290e1-94f7-49b7-87c4-3cdf41d18c36 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:28.951552 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc290e1-94f7-49b7-87c4-3cdf41d18c36 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:28.951664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc290e1-94f7-49b7-87c4-3cdf41d18c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:29.026772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-0fc290e1-94f7-49b7-87c4-3cdf41d18c36 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:29.028584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0fc290e1-94f7-49b7-87c4-3cdf41d18c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:29.124086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 16:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:29.278378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b76fa1-174b-40b4-89d5-ba1181815933 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:29.279920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c7b76fa1-174b-40b4-89d5-ba1181815933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:29.362614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:29.424167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b76fa1-174b-40b4-89d5-ba1181815933 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:29.534225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460973d-3ffc-4cee-a60a-964f7a2e353b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:29.534742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460973d-3ffc-4cee-a60a-964f7a2e353b service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:29.535019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460973d-3ffc-4cee-a60a-964f7a2e353b service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:29.535288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460973d-3ffc-4cee-a60a-964f7a2e353b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:29.536970 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0460973d-3ffc-4cee-a60a-964f7a2e353b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:29.584779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:29.628051 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460973d-3ffc-4cee-a60a-964f7a2e353b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:29.629676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460973d-3ffc-4cee-a60a-964f7a2e353b service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:29.647873 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 717/1428] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:30.449461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0df51-a488-4d86-b89d-755d599f4446 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:30.450032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0df51-a488-4d86-b89d-755d599f4446 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:30.450032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0df51-a488-4d86-b89d-755d599f4446 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:30.451402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 16:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:30.456679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84288f52-b3fb-4ff0-9ea7-6f05aaeb722c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:30.457110 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84288f52-b3fb-4ff0-9ea7-6f05aaeb722c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:30.457294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84288f52-b3fb-4ff0-9ea7-6f05aaeb722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:30.533867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-84288f52-b3fb-4ff0-9ea7-6f05aaeb722c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:30.535076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-84288f52-b3fb-4ff0-9ea7-6f05aaeb722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:30.631380 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 16:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:30.780648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0df51-a488-4d86-b89d-755d599f4446 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:30.782188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-90b0df51-a488-4d86-b89d-755d599f4446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:30.832082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:30.931678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0df51-a488-4d86-b89d-755d599f4446 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:31.049346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e15375-42af-4de2-800f-2150c4c18b4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:31.049897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e15375-42af-4de2-800f-2150c4c18b4f service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:31.050176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e15375-42af-4de2-800f-2150c4c18b4f service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:31.050447 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e15375-42af-4de2-800f-2150c4c18b4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:31.052418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c2e15375-42af-4de2-800f-2150c4c18b4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:31.059589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:31.144516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e15375-42af-4de2-800f-2150c4c18b4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:31.146171 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e15375-42af-4de2-800f-2150c4c18b4f service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:31.164192 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 713/1431] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:31.970857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2872991-b775-4d46-a0cd-28d3ecc62d50 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:31.971434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2872991-b775-4d46-a0cd-28d3ecc62d50 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:31.971434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2872991-b775-4d46-a0cd-28d3ecc62d50 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:31.972696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 16:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:31.977505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e3aa3-5ba2-4e9b-a6f3-89eb00233bc4 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:31.977948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e3aa3-5ba2-4e9b-a6f3-89eb00233bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:31.978080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e3aa3-5ba2-4e9b-a6f3-89eb00233bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:32.052161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b88e3aa3-5ba2-4e9b-a6f3-89eb00233bc4 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:32.053384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b88e3aa3-5ba2-4e9b-a6f3-89eb00233bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:32.123381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:32.151973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 16:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:32.301769 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2872991-b775-4d46-a0cd-28d3ecc62d50 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:32.303328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b2872991-b775-4d46-a0cd-28d3ecc62d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:32.320817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:32.446935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2872991-b775-4d46-a0cd-28d3ecc62d50 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:32.534941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:32.561352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649e449-5766-4dc3-b908-85efe4f40115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:32.561902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649e449-5766-4dc3-b908-85efe4f40115 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:32.562178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649e449-5766-4dc3-b908-85efe4f40115 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:32.562448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649e449-5766-4dc3-b908-85efe4f40115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:32.564040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b649e449-5766-4dc3-b908-85efe4f40115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:32.654495 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649e449-5766-4dc3-b908-85efe4f40115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:32.656111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649e449-5766-4dc3-b908-85efe4f40115 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:32.674445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 720/1434] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:33.476130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ec212-212d-467e-829c-5ea766c850d9 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:33.476705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ec212-212d-467e-829c-5ea766c850d9 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:33.476869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ec212-212d-467e-829c-5ea766c850d9 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:33.478158 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 16:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:33.483975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be962e84-0dfe-4f2a-ad91-747be95c5887 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:33.484469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be962e84-0dfe-4f2a-ad91-747be95c5887 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:33.485538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be962e84-0dfe-4f2a-ad91-747be95c5887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:33.561363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-be962e84-0dfe-4f2a-ad91-747be95c5887 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:33.562580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-be962e84-0dfe-4f2a-ad91-747be95c5887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:33.604248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:33.662523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 16:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:33.812087 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ec212-212d-467e-829c-5ea766c850d9 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:33.813702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-799ec212-212d-467e-829c-5ea766c850d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:34.075035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ec212-212d-467e-829c-5ea766c850d9 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:34.129462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:34.187455 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78439200-0755-4639-b087-2eeb6c4c503a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:34.188061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78439200-0755-4639-b087-2eeb6c4c503a service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:34.188338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78439200-0755-4639-b087-2eeb6c4c503a service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:34.188609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78439200-0755-4639-b087-2eeb6c4c503a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:34.190245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-78439200-0755-4639-b087-2eeb6c4c503a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:34.281768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78439200-0755-4639-b087-2eeb6c4c503a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:34.283360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78439200-0755-4639-b087-2eeb6c4c503a service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:34.301980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 716/1437] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:35.112345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab0e48-9afc-4931-9c92-8cdf44b68f81 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:35.112911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab0e48-9afc-4931-9c92-8cdf44b68f81 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:35.112911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab0e48-9afc-4931-9c92-8cdf44b68f81 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:35.114277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 16:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:35.119257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38305f-532b-4cad-9299-a1611fee8aaa None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:35.119726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38305f-532b-4cad-9299-a1611fee8aaa None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:35.119825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38305f-532b-4cad-9299-a1611fee8aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:35.193861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-6b38305f-532b-4cad-9299-a1611fee8aaa None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:35.195067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6b38305f-532b-4cad-9299-a1611fee8aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:35.197074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:35.292242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 16:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:35.449862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab0e48-9afc-4931-9c92-8cdf44b68f81 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:35.451230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a6ab0e48-9afc-4931-9c92-8cdf44b68f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:35.596387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab0e48-9afc-4931-9c92-8cdf44b68f81 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:35.622782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:35.709214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299ab2b-1ad4-46db-9af0-a0628e7ef58c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:35.709749 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299ab2b-1ad4-46db-9af0-a0628e7ef58c service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:35.710034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299ab2b-1ad4-46db-9af0-a0628e7ef58c service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:35.710303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299ab2b-1ad4-46db-9af0-a0628e7ef58c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:35.711909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8299ab2b-1ad4-46db-9af0-a0628e7ef58c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:35.799952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299ab2b-1ad4-46db-9af0-a0628e7ef58c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:35.801581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299ab2b-1ad4-46db-9af0-a0628e7ef58c service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:35.819820 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 723/1440] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:37.654012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fec5c-eece-4454-9705-74269776f137 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:37.654514 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fec5c-eece-4454-9705-74269776f137 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:37.654514 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fec5c-eece-4454-9705-74269776f137 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:37.655830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:37.661150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafb9d3-6066-432b-8b40-489e8a5552fb None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:37.661588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafb9d3-6066-432b-8b40-489e8a5552fb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:37.661697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafb9d3-6066-432b-8b40-489e8a5552fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:37.702977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:37.728128 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3aafb9d3-6066-432b-8b40-489e8a5552fb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:37.729376 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3aafb9d3-6066-432b-8b40-489e8a5552fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:37.825759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:38.110371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fec5c-eece-4454-9705-74269776f137 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:38.111935 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a07fec5c-eece-4454-9705-74269776f137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:38.212427 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fec5c-eece-4454-9705-74269776f137 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:38.243915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:38.286317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e50db3-80d8-408e-bae0-3351780b52ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:38.286917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e50db3-80d8-408e-bae0-3351780b52ea service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:38.287215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e50db3-80d8-408e-bae0-3351780b52ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:38.287480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e50db3-80d8-408e-bae0-3351780b52ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:38.289089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-09e50db3-80d8-408e-bae0-3351780b52ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:38.395850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e50db3-80d8-408e-bae0-3351780b52ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:38.397237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e50db3-80d8-408e-bae0-3351780b52ea service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:38.415405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 719/1443] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:40.464844 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32bfe7-be2f-483f-a6c7-5d4e45951f4f None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:40.466062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32bfe7-be2f-483f-a6c7-5d4e45951f4f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:40.466062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32bfe7-be2f-483f-a6c7-5d4e45951f4f None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:40.467372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:40.472645 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0de768-cca5-4ccb-887d-2a9cfae7a711 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:40.473109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0de768-cca5-4ccb-887d-2a9cfae7a711 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:40.473224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0de768-cca5-4ccb-887d-2a9cfae7a711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:40.539215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-0f0de768-cca5-4ccb-887d-2a9cfae7a711 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:40.540429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0f0de768-cca5-4ccb-887d-2a9cfae7a711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:40.553230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:40.638228 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:40.923564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32bfe7-be2f-483f-a6c7-5d4e45951f4f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:40.930951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3e32bfe7-be2f-483f-a6c7-5d4e45951f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:41.028545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32bfe7-be2f-483f-a6c7-5d4e45951f4f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:41.040390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:41.102891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c3663-e5f4-4a26-a038-ea0aaa46b547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:41.103507 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c3663-e5f4-4a26-a038-ea0aaa46b547 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:41.103830 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c3663-e5f4-4a26-a038-ea0aaa46b547 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:41.104159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c3663-e5f4-4a26-a038-ea0aaa46b547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:41.106385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e72c3663-e5f4-4a26-a038-ea0aaa46b547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:41.215450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c3663-e5f4-4a26-a038-ea0aaa46b547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:41.216844 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c3663-e5f4-4a26-a038-ea0aaa46b547 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:41.234585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 726/1446] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:43.431755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baac123-6648-41f8-bea4-b4707cd563a2 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:43.432310 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baac123-6648-41f8-bea4-b4707cd563a2 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:43.432310 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baac123-6648-41f8-bea4-b4707cd563a2 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:43.433687 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:43.439665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1212-5df8-41c4-a240-ef7b47a378f5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:43.440096 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1212-5df8-41c4-a240-ef7b47a378f5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:43.440225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1212-5df8-41c4-a240-ef7b47a378f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:43.490058 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:43.509017 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-fa8c1212-5df8-41c4-a240-ef7b47a378f5 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:43.510256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fa8c1212-5df8-41c4-a240-ef7b47a378f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:43.607719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:43.894262 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baac123-6648-41f8-bea4-b4707cd563a2 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:43.895901 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-1baac123-6648-41f8-bea4-b4707cd563a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:43.998972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baac123-6648-41f8-bea4-b4707cd563a2 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:44.038124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:44.070902 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97202a-0a4f-4db1-81d9-ee3f5aeeadd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:44.071530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97202a-0a4f-4db1-81d9-ee3f5aeeadd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:44.071803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97202a-0a4f-4db1-81d9-ee3f5aeeadd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:44.072071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97202a-0a4f-4db1-81d9-ee3f5aeeadd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:44.073674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2f97202a-0a4f-4db1-81d9-ee3f5aeeadd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:44.180781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97202a-0a4f-4db1-81d9-ee3f5aeeadd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:44.182143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97202a-0a4f-4db1-81d9-ee3f5aeeadd4 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:44.201294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 722/1449] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:46.206739 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311b66f-d096-435a-b35b-551e90f7abe6 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:46.207301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311b66f-d096-435a-b35b-551e90f7abe6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:46.207301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311b66f-d096-435a-b35b-551e90f7abe6 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:46.208661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:46.214115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2631c4d-ba0f-4a79-bab7-eb2ba1582791 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:46.214561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2631c4d-ba0f-4a79-bab7-eb2ba1582791 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:46.214670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2631c4d-ba0f-4a79-bab7-eb2ba1582791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:46.282213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d2631c4d-ba0f-4a79-bab7-eb2ba1582791 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:46.283417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d2631c4d-ba0f-4a79-bab7-eb2ba1582791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:46.304052 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:46.380462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:46.667404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311b66f-d096-435a-b35b-551e90f7abe6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:46.669031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b311b66f-d096-435a-b35b-551e90f7abe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:46.770556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311b66f-d096-435a-b35b-551e90f7abe6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:46.790343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:46.846236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afe8c8-262b-4229-b2bb-3371d551ecd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:46.846767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afe8c8-262b-4229-b2bb-3371d551ecd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:46.847047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afe8c8-262b-4229-b2bb-3371d551ecd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:46.847320 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afe8c8-262b-4229-b2bb-3371d551ecd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:46.848961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b2afe8c8-262b-4229-b2bb-3371d551ecd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:46.959830 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afe8c8-262b-4229-b2bb-3371d551ecd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:46.961327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afe8c8-262b-4229-b2bb-3371d551ecd8 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:46.979824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 729/1452] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:48.936452 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9184-f641-4c4e-b427-de5134a3027f None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:48.936987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9184-f641-4c4e-b427-de5134a3027f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:48.936987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9184-f641-4c4e-b427-de5134a3027f None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:48.938317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:25:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:48.943420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fbf8e-8885-4646-be2a-85adfc539e3f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:48.943821 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fbf8e-8885-4646-be2a-85adfc539e3f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:48.943971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fbf8e-8885-4646-be2a-85adfc539e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:49.008462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:49.011230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a28fbf8e-8885-4646-be2a-85adfc539e3f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:49.013354 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a28fbf8e-8885-4646-be2a-85adfc539e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:49.110951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:49.393210 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9184-f641-4c4e-b427-de5134a3027f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:49.394744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-152e9184-f641-4c4e-b427-de5134a3027f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:49.496921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9184-f641-4c4e-b427-de5134a3027f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:49.545302 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:49.569459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82694c45-46e3-48c2-bb60-fdd4db49da3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:49.570064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82694c45-46e3-48c2-bb60-fdd4db49da3c service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:49.570346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82694c45-46e3-48c2-bb60-fdd4db49da3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:49.570618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82694c45-46e3-48c2-bb60-fdd4db49da3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:49.572213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-82694c45-46e3-48c2-bb60-fdd4db49da3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:49.682602 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82694c45-46e3-48c2-bb60-fdd4db49da3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:49.684021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82694c45-46e3-48c2-bb60-fdd4db49da3c service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:49.702342 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 725/1455] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:51.641673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0661222-778d-4067-865e-17ec4be15c77 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:51.642229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0661222-778d-4067-865e-17ec4be15c77 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:51.642229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0661222-778d-4067-865e-17ec4be15c77 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:51.643588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:51.649093 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063923a-4f07-4fa4-b573-282a3b2a7be3 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:51.649548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063923a-4f07-4fa4-b573-282a3b2a7be3 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:51.649646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063923a-4f07-4fa4-b573-282a3b2a7be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:51.714012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d063923a-4f07-4fa4-b573-282a3b2a7be3 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:51.715208 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d063923a-4f07-4fa4-b573-282a3b2a7be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:51.743813 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:51.812653 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:52.093047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0661222-778d-4067-865e-17ec4be15c77 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:52.094574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f0661222-778d-4067-865e-17ec4be15c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:52.197440 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0661222-778d-4067-865e-17ec4be15c77 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:52.225942 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:52.270289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e970e9-b77e-4aee-bb05-9610699a2d5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:52.270817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e970e9-b77e-4aee-bb05-9610699a2d5c service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:52.271094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e970e9-b77e-4aee-bb05-9610699a2d5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:52.271367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e970e9-b77e-4aee-bb05-9610699a2d5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:52.272979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f5e970e9-b77e-4aee-bb05-9610699a2d5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:52.380560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e970e9-b77e-4aee-bb05-9610699a2d5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:52.382040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e970e9-b77e-4aee-bb05-9610699a2d5c service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:52.400743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 732/1458] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:54.451536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8d349-ef86-484e-9961-6ba7539f0af0 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:54.452103 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8d349-ef86-484e-9961-6ba7539f0af0 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:54.452103 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8d349-ef86-484e-9961-6ba7539f0af0 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:54.453433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:25:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:54.458653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8737c2d-01e1-419f-82a6-fc5efeca8f43 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:54.459091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8737c2d-01e1-419f-82a6-fc5efeca8f43 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:54.459202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8737c2d-01e1-419f-82a6-fc5efeca8f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:54.525006 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e8737c2d-01e1-419f-82a6-fc5efeca8f43 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:54.526225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e8737c2d-01e1-419f-82a6-fc5efeca8f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:54.534613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:54.627941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:54.913792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8d349-ef86-484e-9961-6ba7539f0af0 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:54.915341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7ff8d349-ef86-484e-9961-6ba7539f0af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:55.018713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8d349-ef86-484e-9961-6ba7539f0af0 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:55.073371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:55.091642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6262a-61e6-49c1-8e54-1272416d499b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:55.092246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6262a-61e6-49c1-8e54-1272416d499b service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:55.092525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6262a-61e6-49c1-8e54-1272416d499b service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:55.092825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6262a-61e6-49c1-8e54-1272416d499b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:55.094422 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d1f6262a-61e6-49c1-8e54-1272416d499b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:55.202293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6262a-61e6-49c1-8e54-1272416d499b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:55.203637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6262a-61e6-49c1-8e54-1272416d499b service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:55.222061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 728/1461] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:57.251820 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a88a1c-257d-41cb-b89e-45869a965c0e None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:57.252364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a88a1c-257d-41cb-b89e-45869a965c0e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:57.252364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a88a1c-257d-41cb-b89e-45869a965c0e None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:57.253774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:57.259237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54d93f-11bb-4670-935b-578fd39f178f None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:57.259678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54d93f-11bb-4670-935b-578fd39f178f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:57.259783 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54d93f-11bb-4670-935b-578fd39f178f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:57.335751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-4f54d93f-11bb-4670-935b-578fd39f178f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:25:57.337062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-4f54d93f-11bb-4670-935b-578fd39f178f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:57.379637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:57.441637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:25:57.727108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a88a1c-257d-41cb-b89e-45869a965c0e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:25:57.728650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-64a88a1c-257d-41cb-b89e-45869a965c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:57.832394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a88a1c-257d-41cb-b89e-45869a965c0e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:25:57.870164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:25:57.905886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc495925-458e-4318-b910-817571ba5bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:25:57.906542 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc495925-458e-4318-b910-817571ba5bce service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:57.906768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc495925-458e-4318-b910-817571ba5bce service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:57.907033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc495925-458e-4318-b910-817571ba5bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:57.908626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fc495925-458e-4318-b910-817571ba5bce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:25:58.015692 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc495925-458e-4318-b910-817571ba5bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:25:58.017172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc495925-458e-4318-b910-817571ba5bce service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:25:58.034694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 735/1464] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:25:59.949383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bda448-a944-46c5-a84c-58db568a6b85 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:59.949886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bda448-a944-46c5-a84c-58db568a6b85 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:59.949886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bda448-a944-46c5-a84c-58db568a6b85 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:25:59.951731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:25:59.957092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb50ea-2181-4b7f-8943-2d7fddb2ec76 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:25:59.957477 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb50ea-2181-4b7f-8943-2d7fddb2ec76 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:25:59.957665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb50ea-2181-4b7f-8943-2d7fddb2ec76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:00.023284 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2fcb50ea-2181-4b7f-8943-2d7fddb2ec76 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:26:00.024511 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2fcb50ea-2181-4b7f-8943-2d7fddb2ec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:00.042280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:00.121293 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:26:00.399502 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bda448-a944-46c5-a84c-58db568a6b85 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:26:00.401090 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-80bda448-a944-46c5-a84c-58db568a6b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:00.503852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bda448-a944-46c5-a84c-58db568a6b85 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:26:00.569165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:00.576542 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22334e8-ab6b-4c8a-ab70-471f809092e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:26:00.577197 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22334e8-ab6b-4c8a-ab70-471f809092e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:00.577476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22334e8-ab6b-4c8a-ab70-471f809092e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:00.577746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22334e8-ab6b-4c8a-ab70-471f809092e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:00.579323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d22334e8-ab6b-4c8a-ab70-471f809092e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:00.686921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22334e8-ab6b-4c8a-ab70-471f809092e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:26:00.688391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22334e8-ab6b-4c8a-ab70-471f809092e4 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:26:00.706287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 731/1467] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:26:02.606341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342fabf-b5e4-4353-ab0e-e88f943d6287 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:02.607005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342fabf-b5e4-4353-ab0e-e88f943d6287 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:02.607005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342fabf-b5e4-4353-ab0e-e88f943d6287 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:02.608283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:26:02.613421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7e86a-05c3-4b7b-9e14-0240de4222d6 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:02.613820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7e86a-05c3-4b7b-9e14-0240de4222d6 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:02.613961 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7e86a-05c3-4b7b-9e14-0240de4222d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:02.679770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-26f7e86a-05c3-4b7b-9e14-0240de4222d6 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:26:02.681000 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-26f7e86a-05c3-4b7b-9e14-0240de4222d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:02.728788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:02.776672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:26:03.056216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342fabf-b5e4-4353-ab0e-e88f943d6287 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:26:03.057836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5342fabf-b5e4-4353-ab0e-e88f943d6287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:03.161592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342fabf-b5e4-4353-ab0e-e88f943d6287 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:26:03.208514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:03.235234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b754623-c082-4e7e-a060-aef2d37456aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:26:03.235836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b754623-c082-4e7e-a060-aef2d37456aa service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:03.236114 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b754623-c082-4e7e-a060-aef2d37456aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:03.236391 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b754623-c082-4e7e-a060-aef2d37456aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:03.238058 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7b754623-c082-4e7e-a060-aef2d37456aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:03.345681 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b754623-c082-4e7e-a060-aef2d37456aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:26:03.347047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b754623-c082-4e7e-a060-aef2d37456aa service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:26:03.365505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 738/1470] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:26:12.774914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=22 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:13.363240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=66 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:22.834882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95159c-ecee-43c1-92f7-45e45c038b71 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:22.835395 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95159c-ecee-43c1-92f7-45e45c038b71 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:22.835395 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95159c-ecee-43c1-92f7-45e45c038b71 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:22.836990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 733/1471] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:26:22.842932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b86b09-cf7f-4147-9c1c-768565187b86 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:22.843338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b86b09-cf7f-4147-9c1c-768565187b86 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:22.843512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b86b09-cf7f-4147-9c1c-768565187b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:22.909126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-03b86b09-cf7f-4147-9c1c-768565187b86 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:26:22.910343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-03b86b09-cf7f-4147-9c1c-768565187b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:23.004859 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 739/1472] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:26:23.304793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9edb9-6621-4126-a6bd-27b28f3b44c2 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:23.305081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9edb9-6621-4126-a6bd-27b28f3b44c2 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:23.305353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9edb9-6621-4126-a6bd-27b28f3b44c2 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:23.306680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 734/1473] 10.222.0.55 () {54 vars in 967 bytes} [Fri Aug 19 16:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:26:23.312200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6645b-ebc0-4d07-a3ea-b71bb63035f0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:23.312486 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6645b-ebc0-4d07-a3ea-b71bb63035f0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:23.312773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6645b-ebc0-4d07-a3ea-b71bb63035f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:23.387802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-aea6645b-ebc0-4d07-a3ea-b71bb63035f0 None None] MFA Rules not processed for user `748ad823b944441d907994cda6d8ac7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:26:23.389102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-aea6645b-ebc0-4d07-a3ea-b71bb63035f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:23.409422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:23.464771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 740/1474] 10.222.0.55 () {58 vars in 1064 bytes} [Fri Aug 19 16:26:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:26:23.472333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9edb9-6621-4126-a6bd-27b28f3b44c2 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:26:23.473881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-92e9edb9-6621-4126-a6bd-27b28f3b44c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:23.575335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9edb9-6621-4126-a6bd-27b28f3b44c2 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:26:23.646651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f584a8a-dc7f-4fbd-a693-67d71c37a8ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '748ad823b944441d907994cda6d8ac7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:26:23.647254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f584a8a-dc7f-4fbd-a693-67d71c37a8ff service glance] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:23.647529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f584a8a-dc7f-4fbd-a693-67d71c37a8ff service glance] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:23.647796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f584a8a-dc7f-4fbd-a693-67d71c37a8ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:23.649475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5f584a8a-dc7f-4fbd-a693-67d71c37a8ff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:23.703501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:23.757739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f584a8a-dc7f-4fbd-a693-67d71c37a8ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:26:23.759167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f584a8a-dc7f-4fbd-a693-67d71c37a8ff service glance] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:26:23.777307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 735/1475] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:26:25.756227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b8fe7-0c56-454a-8b61-779a353d8dc8 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:25.756800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b8fe7-0c56-454a-8b61-779a353d8dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:25.756800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b8fe7-0c56-454a-8b61-779a353d8dc8 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:25.758203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 741/1476] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:26:25.764044 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af23316-b782-4dd7-9ab5-7edc42182c08 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:25.764539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af23316-b782-4dd7-9ab5-7edc42182c08 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:25.764785 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af23316-b782-4dd7-9ab5-7edc42182c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:25.830274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-5af23316-b782-4dd7-9ab5-7edc42182c08 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:26:25.831480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5af23316-b782-4dd7-9ab5-7edc42182c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:25.928201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 736/1477] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:26:26.202625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b8fe7-0c56-454a-8b61-779a353d8dc8 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:26:26.204126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a70b8fe7-0c56-454a-8b61-779a353d8dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:26.331483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b8fe7-0c56-454a-8b61-779a353d8dc8 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:26:26.380273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:26.424415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd8373-f6f7-4586-b4a3-1614bdedefd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:26:26.425057 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd8373-f6f7-4586-b4a3-1614bdedefd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:26.425342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd8373-f6f7-4586-b4a3-1614bdedefd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:26.425616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd8373-f6f7-4586-b4a3-1614bdedefd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:26.426252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd8373-f6f7-4586-b4a3-1614bdedefd8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:26:26.428027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-efbd8373-f6f7-4586-b4a3-1614bdedefd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:26:26.428027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Aug 19 16:26:26.428027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: Aug 19 16:26:26.429203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd8373-f6f7-4586-b4a3-1614bdedefd8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:26:26.448605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 742/1478] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Aug 19 16:26:26 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:26:28.344296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168a1fb-08ac-43f5-b777-7a0939d9ac23 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:28.344857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168a1fb-08ac-43f5-b777-7a0939d9ac23 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:28.344857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168a1fb-08ac-43f5-b777-7a0939d9ac23 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:28.346246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 737/1479] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:26:28.351289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a921494-e2db-4956-b4a8-6a35ce23a778 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:28.351718 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a921494-e2db-4956-b4a8-6a35ce23a778 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:28.351841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a921494-e2db-4956-b4a8-6a35ce23a778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:28.417576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2a921494-e2db-4956-b4a8-6a35ce23a778 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:26:28.418787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2a921494-e2db-4956-b4a8-6a35ce23a778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:28.513222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 743/1480] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:26:28.814642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168a1fb-08ac-43f5-b777-7a0939d9ac23 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:26:28.816069 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f168a1fb-08ac-43f5-b777-7a0939d9ac23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:28.825654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:28.960059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168a1fb-08ac-43f5-b777-7a0939d9ac23 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:26:29.040928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:29.074930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21969b3-1a44-4101-b651-1f23335a886c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:26:29.075535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21969b3-1a44-4101-b651-1f23335a886c service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:29.075816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21969b3-1a44-4101-b651-1f23335a886c service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:29.076090 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21969b3-1a44-4101-b651-1f23335a886c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:29.078020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c21969b3-1a44-4101-b651-1f23335a886c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:29.167187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21969b3-1a44-4101-b651-1f23335a886c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:26:29.168796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21969b3-1a44-4101-b651-1f23335a886c service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:26:29.187963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 738/1481] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:26:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:26:31.251435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc30d2-198f-455b-be3c-006864901b38 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:31.252007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc30d2-198f-455b-be3c-006864901b38 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:31.252007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc30d2-198f-455b-be3c-006864901b38 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:31.253367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 744/1482] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:26:31.258424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae9940-56a8-4a51-a0be-d13f9b585289 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:31.258863 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae9940-56a8-4a51-a0be-d13f9b585289 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:31.258988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae9940-56a8-4a51-a0be-d13f9b585289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:31.324038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-e7ae9940-56a8-4a51-a0be-d13f9b585289 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:26:31.325293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e7ae9940-56a8-4a51-a0be-d13f9b585289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:31.372935 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:31.426675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 739/1483] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:26:31.711515 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc30d2-198f-455b-be3c-006864901b38 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:26:31.713424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-34cc30d2-198f-455b-be3c-006864901b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:31.737439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:31.819523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc30d2-198f-455b-be3c-006864901b38 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:26:31.893622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c3768-6b5f-4a79-9286-275edf874687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:26:31.894232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c3768-6b5f-4a79-9286-275edf874687 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:31.894515 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c3768-6b5f-4a79-9286-275edf874687 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:31.894787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c3768-6b5f-4a79-9286-275edf874687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:31.896397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-701c3768-6b5f-4a79-9286-275edf874687 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:31.971805 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:32.006747 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c3768-6b5f-4a79-9286-275edf874687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:26:32.008197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c3768-6b5f-4a79-9286-275edf874687 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:26:32.026665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 745/1484] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:26:33.957388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cff1a2-02d0-41d2-aedf-635cd2821b8f None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:33.957916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cff1a2-02d0-41d2-aedf-635cd2821b8f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:33.957916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cff1a2-02d0-41d2-aedf-635cd2821b8f None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:33.959180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 740/1485] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:26:33.964539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d305b-cea0-40b4-ada0-2a68b666fc92 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:33.965025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d305b-cea0-40b4-ada0-2a68b666fc92 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:33.965129 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d305b-cea0-40b4-ada0-2a68b666fc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:34.030714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6e6d305b-cea0-40b4-ada0-2a68b666fc92 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:26:34.031907 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6e6d305b-cea0-40b4-ada0-2a68b666fc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:34.125759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 746/1486] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:26:34.411505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cff1a2-02d0-41d2-aedf-635cd2821b8f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:26:34.413073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f4cff1a2-02d0-41d2-aedf-635cd2821b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:34.515077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cff1a2-02d0-41d2-aedf-635cd2821b8f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:26:34.585032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:34.587039 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7125369-5fad-4799-bffe-34d36b8ff590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:26:34.587654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7125369-5fad-4799-bffe-34d36b8ff590 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:34.588108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7125369-5fad-4799-bffe-34d36b8ff590 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:34.588435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7125369-5fad-4799-bffe-34d36b8ff590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:34.590506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d7125369-5fad-4799-bffe-34d36b8ff590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:34.699343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7125369-5fad-4799-bffe-34d36b8ff590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:26:34.700802 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7125369-5fad-4799-bffe-34d36b8ff590 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:26:34.719256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 741/1487] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:26:36.801263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303cd21-8fe1-45da-9907-ddc494d6309c None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:36.801852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303cd21-8fe1-45da-9907-ddc494d6309c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:36.801852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303cd21-8fe1-45da-9907-ddc494d6309c None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:36.803170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 747/1488] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:26:36.809126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09416eae-96fb-4298-8fc3-617e369d2064 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:36.809126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09416eae-96fb-4298-8fc3-617e369d2064 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:36.809763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09416eae-96fb-4298-8fc3-617e369d2064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:36.876506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-09416eae-96fb-4298-8fc3-617e369d2064 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:26:36.877656 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-09416eae-96fb-4298-8fc3-617e369d2064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:36.931702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:36.977375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 742/1489] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:26:37.272920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303cd21-8fe1-45da-9907-ddc494d6309c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:26:37.274497 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6303cd21-8fe1-45da-9907-ddc494d6309c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:37.310582 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:37.379684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303cd21-8fe1-45da-9907-ddc494d6309c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:26:37.453226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10172b7b-2d02-46b0-9ea4-a37c5ddbcecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:26:37.453832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10172b7b-2d02-46b0-9ea4-a37c5ddbcecd service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:37.454115 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10172b7b-2d02-46b0-9ea4-a37c5ddbcecd service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:37.454388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10172b7b-2d02-46b0-9ea4-a37c5ddbcecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:37.456002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-10172b7b-2d02-46b0-9ea4-a37c5ddbcecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:37.536288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:37.566877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10172b7b-2d02-46b0-9ea4-a37c5ddbcecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:26:37.568327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10172b7b-2d02-46b0-9ea4-a37c5ddbcecd service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:26:37.586619 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 748/1490] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:26:39.697739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c14a9-2968-4e73-9319-de23fee76168 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:39.698262 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c14a9-2968-4e73-9319-de23fee76168 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:39.698262 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c14a9-2968-4e73-9319-de23fee76168 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:39.699624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 743/1491] 10.222.0.55 () {54 vars in 927 bytes} [Fri Aug 19 16:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:26:39.705481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bcfef-a95d-4444-bfd2-b6ea21735dad None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:39.705890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bcfef-a95d-4444-bfd2-b6ea21735dad None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:39.706032 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bcfef-a95d-4444-bfd2-b6ea21735dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:39.771858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5e5bcfef-a95d-4444-bfd2-b6ea21735dad None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:26:39.773106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5e5bcfef-a95d-4444-bfd2-b6ea21735dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:39.868602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 749/1492] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Aug 19 16:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:26:40.163681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c14a9-2968-4e73-9319-de23fee76168 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:26:40.165267 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d71c14a9-2968-4e73-9319-de23fee76168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:40.269074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c14a9-2968-4e73-9319-de23fee76168 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:26:40.340620 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616fefd-2846-4354-aa77-4a537e552d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:26:40.341281 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616fefd-2846-4354-aa77-4a537e552d06 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:26:40.341563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616fefd-2846-4354-aa77-4a537e552d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:26:40.341836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616fefd-2846-4354-aa77-4a537e552d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:26:40.343994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8616fefd-2846-4354-aa77-4a537e552d06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:26:40.353235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:40.455478 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616fefd-2846-4354-aa77-4a537e552d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:26:40.456881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616fefd-2846-4354-aa77-4a537e552d06 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:26:40.476036 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 744/1493] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:26:49.866812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=82 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:26:50.473719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=50 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:28:08.465444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb34e49-aec2-402e-83ab-bb921dd6884b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:08.466124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb34e49-aec2-402e-83ab-bb921dd6884b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:08.466124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb34e49-aec2-402e-83ab-bb921dd6884b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:08.541038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bbb34e49-aec2-402e-83ab-bb921dd6884b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:28:08.542255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bbb34e49-aec2-402e-83ab-bb921dd6884b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:08.638982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 750/1494] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:28:08.647323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841e1cd-4616-4cd8-9a6c-1856d127cafb None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:08.647811 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841e1cd-4616-4cd8-9a6c-1856d127cafb None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:08.647924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841e1cd-4616-4cd8-9a6c-1856d127cafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:08.722516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d841e1cd-4616-4cd8-9a6c-1856d127cafb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:28:08.723657 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d841e1cd-4616-4cd8-9a6c-1856d127cafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:08.818222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 745/1495] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:28:08.823004 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb34e49-aec2-402e-83ab-bb921dd6884b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:28:08.824220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bbb34e49-aec2-402e-83ab-bb921dd6884b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:08.835593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb34e49-aec2-402e-83ab-bb921dd6884b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:28:08.896365 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:28:08.931536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273d94e-b112-4236-bf3b-0f9d544c47e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:28:08.932136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273d94e-b112-4236-bf3b-0f9d544c47e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:08.932411 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273d94e-b112-4236-bf3b-0f9d544c47e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:08.932676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273d94e-b112-4236-bf3b-0f9d544c47e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:08.933353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273d94e-b112-4236-bf3b-0f9d544c47e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:28:08.933634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273d94e-b112-4236-bf3b-0f9d544c47e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:28:08.934639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-9273d94e-b112-4236-bf3b-0f9d544c47e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:28:08.934639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Aug 19 16:28:08.934639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: Aug 19 16:28:08.935521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273d94e-b112-4236-bf3b-0f9d544c47e6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:28:08.946511 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 751/1496] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:28:08.955050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841e1cd-4616-4cd8-9a6c-1856d127cafb None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:28:08.964406 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841e1cd-4616-4cd8-9a6c-1856d127cafb None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:28:09.025953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:28:09.059148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c7295-5182-4698-bd6d-d75a7352fc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:28:09.059751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c7295-5182-4698-bd6d-d75a7352fc58 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:09.060020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c7295-5182-4698-bd6d-d75a7352fc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:09.060285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c7295-5182-4698-bd6d-d75a7352fc58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:09.060925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c7295-5182-4698-bd6d-d75a7352fc58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:28:09.062924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: WARNING py.warnings [None req-292c7295-5182-4698-bd6d-d75a7352fc58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:28:09.062924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: warnings.warn(msg) Aug 19 16:28:09.062924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: Aug 19 16:28:09.064119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c7295-5182-4698-bd6d-d75a7352fc58 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:28:09.121541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 746/1497] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:28:08 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:28:09.126276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273d94e-b112-4236-bf3b-0f9d544c47e6 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:28:09.128123 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9273d94e-b112-4236-bf3b-0f9d544c47e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:09.161677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273d94e-b112-4236-bf3b-0f9d544c47e6 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:28:09.253643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267d135-387c-4c9b-9ac3-1d01893efa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:28:09.254240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267d135-387c-4c9b-9ac3-1d01893efa22 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:09.254514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267d135-387c-4c9b-9ac3-1d01893efa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:09.254783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267d135-387c-4c9b-9ac3-1d01893efa22 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:09.255388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267d135-387c-4c9b-9ac3-1d01893efa22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:28:09.256400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-a267d135-387c-4c9b-9ac3-1d01893efa22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:28:09.256400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Aug 19 16:28:09.256400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: Aug 19 16:28:09.257652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267d135-387c-4c9b-9ac3-1d01893efa22 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:28:09.299671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 752/1498] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:28:09 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:28:09.303872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c7295-5182-4698-bd6d-d75a7352fc58 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:28:09.305589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-292c7295-5182-4698-bd6d-d75a7352fc58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:09.367629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c7295-5182-4698-bd6d-d75a7352fc58 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:28:09.449460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:28:09.461224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b33732-08dc-4d45-9774-a9529026a50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:28:09.461853 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b33732-08dc-4d45-9774-a9529026a50f admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:09.462130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b33732-08dc-4d45-9774-a9529026a50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:09.462397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b33732-08dc-4d45-9774-a9529026a50f admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:09.463033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b33732-08dc-4d45-9774-a9529026a50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:28:09.464035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: WARNING py.warnings [None req-d9b33732-08dc-4d45-9774-a9529026a50f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:28:09.464035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: warnings.warn(msg) Aug 19 16:28:09.464035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: Aug 19 16:28:09.465213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b33732-08dc-4d45-9774-a9529026a50f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:28:09.478669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 747/1499] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:28:09.482764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267d135-387c-4c9b-9ac3-1d01893efa22 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:28:09.484346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a267d135-387c-4c9b-9ac3-1d01893efa22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:09.495991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267d135-387c-4c9b-9ac3-1d01893efa22 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:28:09.522537 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:28:09.592659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba843072-9b18-4c70-99b3-d6a786455a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:28:09.593228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba843072-9b18-4c70-99b3-d6a786455a28 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:09.593538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba843072-9b18-4c70-99b3-d6a786455a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:09.593813 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba843072-9b18-4c70-99b3-d6a786455a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:09.594455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba843072-9b18-4c70-99b3-d6a786455a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:28:09.595469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-ba843072-9b18-4c70-99b3-d6a786455a28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:28:09.595469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Aug 19 16:28:09.595469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: Aug 19 16:28:09.596350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba843072-9b18-4c70-99b3-d6a786455a28 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:28:09.608442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 753/1500] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:28:09.612610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b33732-08dc-4d45-9774-a9529026a50f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:28:09.614152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d9b33732-08dc-4d45-9774-a9529026a50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:09.649043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b33732-08dc-4d45-9774-a9529026a50f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:28:09.743763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae6630-e91d-44a3-a8fd-2b88725a69fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:28:09.744420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae6630-e91d-44a3-a8fd-2b88725a69fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:09.744700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae6630-e91d-44a3-a8fd-2b88725a69fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:09.745003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae6630-e91d-44a3-a8fd-2b88725a69fd admin admin] PATH_INFO: `/v3/projects/15f7bdf60f8e42dca35912f3c04f0fd6/users/54569c3cce4f4c5e8b3c30b9a66cae0e/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:09.767686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae6630-e91d-44a3-a8fd-2b88725a69fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=15f7bdf60f8e42dca35912f3c04f0fd6, user_id=54569c3cce4f4c5e8b3c30b9a66cae0e, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:28:09.768738 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: WARNING py.warnings [None req-b1ae6630-e91d-44a3-a8fd-2b88725a69fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:28:09.768738 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: warnings.warn(msg) Aug 19 16:28:09.768738 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: Aug 19 16:28:09.770706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae6630-e91d-44a3-a8fd-2b88725a69fd admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:28:09.794689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 748/1501] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:28:09 2022] PUT /identity/v3/projects/15f7bdf60f8e42dca35912f3c04f0fd6/users/54569c3cce4f4c5e8b3c30b9a66cae0e/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:28:09.807842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b4993-e23e-4ae3-a46f-3f9a7c04893b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:09.808130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b4993-e23e-4ae3-a46f-3f9a7c04893b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:09.808408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b4993-e23e-4ae3-a46f-3f9a7c04893b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:09.865510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-642b4993-e23e-4ae3-a46f-3f9a7c04893b None None] MFA Rules not processed for user `54569c3cce4f4c5e8b3c30b9a66cae0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:28:09.866635 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-642b4993-e23e-4ae3-a46f-3f9a7c04893b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:09.943790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 754/1502] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:28:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:28:09.957429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1f28d-bcb5-411e-a1c0-4cc5d6a39bd2 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:09.957715 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1f28d-bcb5-411e-a1c0-4cc5d6a39bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:09.957988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1f28d-bcb5-411e-a1c0-4cc5d6a39bd2 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:09.959305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 749/1503] 10.222.0.55 () {54 vars in 964 bytes} [Fri Aug 19 16:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:28:09.964490 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a904a-3415-467e-a316-3c8b00fd53f9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:09.964820 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a904a-3415-467e-a316-3c8b00fd53f9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:09.965107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a904a-3415-467e-a316-3c8b00fd53f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:09.996988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:28:10.042679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-838a904a-3415-467e-a316-3c8b00fd53f9 None None] MFA Rules not processed for user `1eaa3980ad8840879e4e426b00e3c56f`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:28:10.043889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-838a904a-3415-467e-a316-3c8b00fd53f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:10.118521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 755/1504] 10.222.0.55 () {58 vars in 1061 bytes} [Fri Aug 19 16:28:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:28:10.126906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1f28d-bcb5-411e-a1c0-4cc5d6a39bd2 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:28:10.128440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a5c1f28d-bcb5-411e-a1c0-4cc5d6a39bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:10.177016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:28:10.231220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1f28d-bcb5-411e-a1c0-4cc5d6a39bd2 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:28:10.305437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ae5fb-6134-4327-aef7-4ff717a081ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:28:10.306052 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ae5fb-6134-4327-aef7-4ff717a081ee service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:10.306353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ae5fb-6134-4327-aef7-4ff717a081ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:10.306630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ae5fb-6134-4327-aef7-4ff717a081ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:10.308294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b70ae5fb-6134-4327-aef7-4ff717a081ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:10.408627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ae5fb-6134-4327-aef7-4ff717a081ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:28:10.410015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ae5fb-6134-4327-aef7-4ff717a081ee service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:28:10.413482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:28:10.428542 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 750/1505] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:28:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:28:10.473671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b392b-73e9-4ac3-8ad4-44b2d7c0c285 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:10.473966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b392b-73e9-4ac3-8ad4-44b2d7c0c285 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:10.474247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b392b-73e9-4ac3-8ad4-44b2d7c0c285 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:10.475552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 756/1506] 10.222.0.55 () {52 vars in 830 bytes} [Fri Aug 19 16:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:28:10.479081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ae5fb-6134-4327-aef7-4ff717a081ee service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:28:10.480309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b70ae5fb-6134-4327-aef7-4ff717a081ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:10.579229 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ae5fb-6134-4327-aef7-4ff717a081ee service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:28:10.671031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:28:10.672158 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:10.672402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:10.672673 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] PATH_INFO: `/v3/users/54569c3cce4f4c5e8b3c30b9a66cae0e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:10.684607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=54569c3cce4f4c5e8b3c30b9a66cae0e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:28:10.685663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: WARNING py.warnings [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:28:10.685663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: warnings.warn(msg) Aug 19 16:28:10.685663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: Aug 19 16:28:10.686877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:28:10.697150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:28:10.753155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54569c3cce4f4c5e8b3c30b9a66cae0e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:28:10.753588 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54569c3cce4f4c5e8b3c30b9a66cae0e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:28:10.754105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54569c3cce4f4c5e8b3c30b9a66cae0e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:28:10.767756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54569c3cce4f4c5e8b3c30b9a66cae0e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:28:10.789679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54569c3cce4f4c5e8b3c30b9a66cae0e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:28:10.804225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 751/1507] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:28:10 2022] DELETE /identity/v3/users/54569c3cce4f4c5e8b3c30b9a66cae0e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:28:10.808885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b796e8-7755-480e-90a9-7c8fcc635685 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:10.809378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b796e8-7755-480e-90a9-7c8fcc635685 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:10.809484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b796e8-7755-480e-90a9-7c8fcc635685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:10.866187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-24b796e8-7755-480e-90a9-7c8fcc635685 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:28:10.867408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-24b796e8-7755-480e-90a9-7c8fcc635685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:10.943164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:28:10.956592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 757/1508] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:28:10.966981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:28:10.968227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:11.069320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174f77a-950d-434e-ba02-0e2a7ac2c4fd admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:28:11.125584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:28:11.147329 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cddaf1-f6ec-4141-b955-1a08da35cd47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:28:11.147953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cddaf1-f6ec-4141-b955-1a08da35cd47 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:11.148231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cddaf1-f6ec-4141-b955-1a08da35cd47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:11.148499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cddaf1-f6ec-4141-b955-1a08da35cd47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:11.150133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-11cddaf1-f6ec-4141-b955-1a08da35cd47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:11.268724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cddaf1-f6ec-4141-b955-1a08da35cd47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:28:11.270190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cddaf1-f6ec-4141-b955-1a08da35cd47 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:28:11.289383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 752/1509] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:28:11.651011 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b796e8-7755-480e-90a9-7c8fcc635685 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:28:11.652736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-24b796e8-7755-480e-90a9-7c8fcc635685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:28:11.666788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b796e8-7755-480e-90a9-7c8fcc635685 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:28:11.774064 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:28:11.774710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:28:11.774989 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:28:11.775328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] PATH_INFO: `/v3/projects/15f7bdf60f8e42dca35912f3c04f0fd6` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:28:11.786751 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=15f7bdf60f8e42dca35912f3c04f0fd6)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:28:11.787803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:28:11.787803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Aug 19 16:28:11.787803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: Aug 19 16:28:11.789049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:28:11.847883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15f7bdf60f8e42dca35912f3c04f0fd6', 'request_id': 'req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:28:11.853762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15f7bdf60f8e42dca35912f3c04f0fd6', 'request_id': 'req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:28:11.867788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15f7bdf60f8e42dca35912f3c04f0fd6', 'request_id': 'req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:28:11.896601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] The token cache is being invalidate because project 15f7bdf60f8e42dca35912f3c04f0fd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:28:11.896927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:28:11.897358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-41b859f5-b0bd-4ec8-b8a5-dad4c821e4bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:28:11.898753 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 758/1510] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:28:11 2022] DELETE /identity/v3/projects/15f7bdf60f8e42dca35912f3c04f0fd6 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:28:21.286620 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=62 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:28:21.894540 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:29:46.552389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657b5a7-07db-4a7d-b396-46460d4014c7 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:29:46.553009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657b5a7-07db-4a7d-b396-46460d4014c7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:29:46.553009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657b5a7-07db-4a7d-b396-46460d4014c7 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:29:46.554322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 16:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:29:46.559067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e28c0-f6c7-4d6e-89c5-334c581b99f4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:29:46.559495 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e28c0-f6c7-4d6e-89c5-334c581b99f4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:29:46.559616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e28c0-f6c7-4d6e-89c5-334c581b99f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:29:46.760250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-881e28c0-f6c7-4d6e-89c5-334c581b99f4 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:29:46.761553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-881e28c0-f6c7-4d6e-89c5-334c581b99f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:29:46.868309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 16:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:29:47.033542 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657b5a7-07db-4a7d-b396-46460d4014c7 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:29:47.035649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f657b5a7-07db-4a7d-b396-46460d4014c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:29:47.159407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657b5a7-07db-4a7d-b396-46460d4014c7 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:29:47.164857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b2c65-5ab4-45fc-8e19-8765e1e5429e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:29:47.165466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b2c65-5ab4-45fc-8e19-8765e1e5429e service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:29:47.165750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b2c65-5ab4-45fc-8e19-8765e1e5429e service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:29:47.166029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b2c65-5ab4-45fc-8e19-8765e1e5429e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:29:47.175021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b2c65-5ab4-45fc-8e19-8765e1e5429e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:29:47.176663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b2c65-5ab4-45fc-8e19-8765e1e5429e service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:29:47.186414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 754/1513] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:29:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:29:48.018820 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a26400-d98f-421f-9a3d-9b3fbdb95c98 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:29:48.019393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a26400-d98f-421f-9a3d-9b3fbdb95c98 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:29:48.019393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a26400-d98f-421f-9a3d-9b3fbdb95c98 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:29:48.020712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 16:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:29:48.025559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dfe08-6b93-4cac-a27e-3c33f70a123c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:29:48.026003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dfe08-6b93-4cac-a27e-3c33f70a123c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:29:48.026107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dfe08-6b93-4cac-a27e-3c33f70a123c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:29:48.045541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f05dfe08-6b93-4cac-a27e-3c33f70a123c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:29:48.046742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f05dfe08-6b93-4cac-a27e-3c33f70a123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:29:48.054480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 16:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:29:48.207529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a26400-d98f-421f-9a3d-9b3fbdb95c98 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:29:48.222888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a26400-d98f-421f-9a3d-9b3fbdb95c98 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:29:48.228357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d708d-90e8-4051-93d6-374e4784e5dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:29:48.228988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d708d-90e8-4051-93d6-374e4784e5dc service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:29:48.229263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d708d-90e8-4051-93d6-374e4784e5dc service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:29:48.229532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d708d-90e8-4051-93d6-374e4784e5dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:29:48.237775 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d708d-90e8-4051-93d6-374e4784e5dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:29:48.239370 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d708d-90e8-4051-93d6-374e4784e5dc service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:29:48.248565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 761/1516] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:29:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:29:50.192032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077785-f836-48ac-8311-63688244fa18 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:29:50.192592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077785-f836-48ac-8311-63688244fa18 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:29:50.192592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077785-f836-48ac-8311-63688244fa18 None None] PATH_INFO: `/v3` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:29:50.193957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 756/1517] 10.222.0.58 () {54 vars in 925 bytes} [Fri Aug 19 16:29:50 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:29:50.202482 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78085509-f250-4f97-bca4-c3a94e1bcf0a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:29:50.202905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78085509-f250-4f97-bca4-c3a94e1bcf0a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:29:50.203033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78085509-f250-4f97-bca4-c3a94e1bcf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:29:50.262002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-78085509-f250-4f97-bca4-c3a94e1bcf0a None None] MFA Rules not processed for user `1dcbe2b128a94b9abbf3a9b25db1e845`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:29:50.263202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-78085509-f250-4f97-bca4-c3a94e1bcf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:29:50.279596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:29:50.334945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 762/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Fri Aug 19 16:29:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:29:58.040884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=56 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:29:59.501913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e7be1-11e2-4aa0-ac41-a6eba34f7691 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:29:59.502560 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e7be1-11e2-4aa0-ac41-a6eba34f7691 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:29:59.502560 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e7be1-11e2-4aa0-ac41-a6eba34f7691 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:29:59.503821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 757/1519] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 16:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:29:59.508629 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f29e8-9f6f-4e3a-9496-2b5e9c72498b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:29:59.509084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f29e8-9f6f-4e3a-9496-2b5e9c72498b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:29:59.509246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f29e8-9f6f-4e3a-9496-2b5e9c72498b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:29:59.529117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d78f29e8-9f6f-4e3a-9496-2b5e9c72498b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:29:59.530333 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d78f29e8-9f6f-4e3a-9496-2b5e9c72498b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:29:59.538898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 763/1520] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 16:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:29:59.692605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e7be1-11e2-4aa0-ac41-a6eba34f7691 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:29:59.708770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e7be1-11e2-4aa0-ac41-a6eba34f7691 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:29:59.714367 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d658b-a869-40d6-9f4a-02fac9803a1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:29:59.714956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d658b-a869-40d6-9f4a-02fac9803a1e service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:29:59.715231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d658b-a869-40d6-9f4a-02fac9803a1e service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:29:59.715500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d658b-a869-40d6-9f4a-02fac9803a1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:29:59.723698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d658b-a869-40d6-9f4a-02fac9803a1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:29:59.725340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d658b-a869-40d6-9f4a-02fac9803a1e service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:29:59.734691 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 758/1521] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:29:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:30:00.555815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e1ac6-bd79-4467-a70f-701fa6d83fe4 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:00.556369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e1ac6-bd79-4467-a70f-701fa6d83fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:00.556369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e1ac6-bd79-4467-a70f-701fa6d83fe4 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:00.557714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 16:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:30:00.562402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a158d9e-2964-4c31-a19f-4c6299193fe2 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:00.562789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a158d9e-2964-4c31-a19f-4c6299193fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:00.563061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a158d9e-2964-4c31-a19f-4c6299193fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:00.581941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-7a158d9e-2964-4c31-a19f-4c6299193fe2 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:30:00.583141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7a158d9e-2964-4c31-a19f-4c6299193fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:30:00.591194 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 16:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:30:00.743006 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e1ac6-bd79-4467-a70f-701fa6d83fe4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:30:00.757504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e1ac6-bd79-4467-a70f-701fa6d83fe4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:30:00.762886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9a44d-c307-4469-8255-c314393acedc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:30:00.763468 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9a44d-c307-4469-8255-c314393acedc service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:00.763744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9a44d-c307-4469-8255-c314393acedc service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:00.764014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9a44d-c307-4469-8255-c314393acedc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:00.772379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9a44d-c307-4469-8255-c314393acedc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:30:00.773993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9a44d-c307-4469-8255-c314393acedc service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:30:00.784438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 765/1524] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:30:01.397061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f66ab-b41e-4d35-9f3e-c44914f7086d None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:01.397253 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f66ab-b41e-4d35-9f3e-c44914f7086d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:01.397519 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f66ab-b41e-4d35-9f3e-c44914f7086d None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:01.398846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 760/1525] 10.222.0.55 () {54 vars in 961 bytes} [Fri Aug 19 16:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:30:01.403716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb38d0-c832-4e15-a63d-31e32261430e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:01.404005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb38d0-c832-4e15-a63d-31e32261430e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:01.404281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb38d0-c832-4e15-a63d-31e32261430e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:01.423836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-57fb38d0-c832-4e15-a63d-31e32261430e None None] MFA Rules not processed for user `1dcbe2b128a94b9abbf3a9b25db1e845`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:30:01.425177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-57fb38d0-c832-4e15-a63d-31e32261430e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:30:01.433899 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 766/1526] 10.222.0.55 () {58 vars in 1058 bytes} [Fri Aug 19 16:30:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:30:01.441961 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f66ab-b41e-4d35-9f3e-c44914f7086d None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:30:01.455213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f66ab-b41e-4d35-9f3e-c44914f7086d None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:30:01.460081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc252f7-476b-4dee-bfa4-34d829fcdd18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:30:01.460666 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc252f7-476b-4dee-bfa4-34d829fcdd18 service placement] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:01.460970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc252f7-476b-4dee-bfa4-34d829fcdd18 service placement] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:01.461242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc252f7-476b-4dee-bfa4-34d829fcdd18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:01.469650 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc252f7-476b-4dee-bfa4-34d829fcdd18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:30:01.471211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc252f7-476b-4dee-bfa4-34d829fcdd18 service placement] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:30:01.480596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 761/1527] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:30:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:30:04.703053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71f14f-8d1c-457f-91ab-a7ce8cabcd36 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:04.703552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71f14f-8d1c-457f-91ab-a7ce8cabcd36 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:04.703665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71f14f-8d1c-457f-91ab-a7ce8cabcd36 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:04.704956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:30:04.710005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec13d4d-5128-460d-b7ea-3ed92db7d746 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:04.710426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec13d4d-5128-460d-b7ea-3ed92db7d746 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:04.710547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec13d4d-5128-460d-b7ea-3ed92db7d746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:04.729522 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3ec13d4d-5128-460d-b7ea-3ed92db7d746 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:30:04.730707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3ec13d4d-5128-460d-b7ea-3ed92db7d746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:30:04.740303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:30:05.025741 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71f14f-8d1c-457f-91ab-a7ce8cabcd36 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:30:05.027560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-aa71f14f-8d1c-457f-91ab-a7ce8cabcd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:30:05.104986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71f14f-8d1c-457f-91ab-a7ce8cabcd36 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:30:05.108514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d34cf-d726-4667-828b-10fe0106890d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:30:05.109816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d34cf-d726-4667-828b-10fe0106890d service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:05.110194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d34cf-d726-4667-828b-10fe0106890d service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:05.110470 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d34cf-d726-4667-828b-10fe0106890d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:05.119163 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d34cf-d726-4667-828b-10fe0106890d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:30:05.120606 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d34cf-d726-4667-828b-10fe0106890d service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:30:05.130495 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 768/1530] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:30:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:30:07.029393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf75e8-0c07-4faa-a8ba-721ebeb88eae None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:07.029959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf75e8-0c07-4faa-a8ba-721ebeb88eae None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:07.029959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf75e8-0c07-4faa-a8ba-721ebeb88eae None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:07.031304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:30:07.036637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9c8ae-f7ba-4ae0-ba06-704528f23251 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:07.037105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9c8ae-f7ba-4ae0-ba06-704528f23251 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:07.037239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9c8ae-f7ba-4ae0-ba06-704528f23251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:07.057967 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a4a9c8ae-f7ba-4ae0-ba06-704528f23251 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:30:07.059201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a4a9c8ae-f7ba-4ae0-ba06-704528f23251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:30:07.067928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:30:07.360395 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf75e8-0c07-4faa-a8ba-721ebeb88eae None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:30:07.371257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf75e8-0c07-4faa-a8ba-721ebeb88eae None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:30:07.375335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec03525-d5f0-470b-957e-f19f30bd0022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:30:07.376051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec03525-d5f0-470b-957e-f19f30bd0022 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:07.376226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec03525-d5f0-470b-957e-f19f30bd0022 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:07.376505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec03525-d5f0-470b-957e-f19f30bd0022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:07.387365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec03525-d5f0-470b-957e-f19f30bd0022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:30:07.388838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec03525-d5f0-470b-957e-f19f30bd0022 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:30:07.398992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 764/1533] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:30:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:30:10.905398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1136ab-c45b-4064-b0f5-1e9822b55372 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:10.905994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1136ab-c45b-4064-b0f5-1e9822b55372 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:10.905994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1136ab-c45b-4064-b0f5-1e9822b55372 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:10.907572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:30:10.912583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d1b1f-e5bd-4947-a8fe-eaa6456f1468 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:10.913054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d1b1f-e5bd-4947-a8fe-eaa6456f1468 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:10.913252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d1b1f-e5bd-4947-a8fe-eaa6456f1468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:10.932193 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-312d1b1f-e5bd-4947-a8fe-eaa6456f1468 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:30:10.933426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-312d1b1f-e5bd-4947-a8fe-eaa6456f1468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:30:10.941809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:30:11.495968 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1136ab-c45b-4064-b0f5-1e9822b55372 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:30:11.497692 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ef1136ab-c45b-4064-b0f5-1e9822b55372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:30:11.568537 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:30:11.584023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1136ab-c45b-4064-b0f5-1e9822b55372 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:30:11.587733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d434c9b-87b8-49c6-9721-a8138b467f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:30:11.588340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d434c9b-87b8-49c6-9721-a8138b467f64 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:11.588907 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d434c9b-87b8-49c6-9721-a8138b467f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:11.589165 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d434c9b-87b8-49c6-9721-a8138b467f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:11.599258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d434c9b-87b8-49c6-9721-a8138b467f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:30:11.600680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d434c9b-87b8-49c6-9721-a8138b467f64 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:30:11.611174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 771/1536] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:30:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:30:13.545286 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad85884-bea2-4a06-8944-b37a9ddfc577 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:13.545827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad85884-bea2-4a06-8944-b37a9ddfc577 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:13.545827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad85884-bea2-4a06-8944-b37a9ddfc577 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:13.547512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 16:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:30:13.552621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944b99a-5f8f-4877-b28d-7d29c848b384 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:13.553046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944b99a-5f8f-4877-b28d-7d29c848b384 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:13.553214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944b99a-5f8f-4877-b28d-7d29c848b384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:13.572257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2944b99a-5f8f-4877-b28d-7d29c848b384 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:30:13.573493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2944b99a-5f8f-4877-b28d-7d29c848b384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:30:13.581866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 16:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:30:14.130396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad85884-bea2-4a06-8944-b37a9ddfc577 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:30:14.132182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8ad85884-bea2-4a06-8944-b37a9ddfc577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:30:14.145662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad85884-bea2-4a06-8944-b37a9ddfc577 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:30:14.149164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d7ca5-bfbb-4ac0-9189-aef034e24cd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:30:14.149755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d7ca5-bfbb-4ac0-9189-aef034e24cd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:30:14.150032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d7ca5-bfbb-4ac0-9189-aef034e24cd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:30:14.150302 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d7ca5-bfbb-4ac0-9189-aef034e24cd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:30:14.159265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d7ca5-bfbb-4ac0-9189-aef034e24cd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:30:14.160664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d7ca5-bfbb-4ac0-9189-aef034e24cd0 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:30:14.170021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 767/1539] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:30:23.567596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=11 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:30:24.165892 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=33 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:41.198343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397517f-f398-4f02-9b2d-a2a09170f311 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.199547 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397517f-f398-4f02-9b2d-a2a09170f311 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.200560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397517f-f398-4f02-9b2d-a2a09170f311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.227375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c397517f-f398-4f02-9b2d-a2a09170f311 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.228623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c397517f-f398-4f02-9b2d-a2a09170f311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.229826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b4804-06f5-4616-a094-2a78016b0448 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.230321 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b4804-06f5-4616-a094-2a78016b0448 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.230410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b4804-06f5-4616-a094-2a78016b0448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.244617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 773/1540] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.267936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ca7b4804-06f5-4616-a094-2a78016b0448 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.271220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ca7b4804-06f5-4616-a094-2a78016b0448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.272556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae41199-70c3-4f21-a1e0-cb872c533102 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.272925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae41199-70c3-4f21-a1e0-cb872c533102 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.273219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae41199-70c3-4f21-a1e0-cb872c533102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.281544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 768/1541] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.290985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871baf3-a08e-43a1-b430-beb62fbf73b1 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.291315 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871baf3-a08e-43a1-b430-beb62fbf73b1 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.291534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871baf3-a08e-43a1-b430-beb62fbf73b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.305597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7ae41199-70c3-4f21-a1e0-cb872c533102 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.306836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7ae41199-70c3-4f21-a1e0-cb872c533102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.326498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d871baf3-a08e-43a1-b430-beb62fbf73b1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.328123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d871baf3-a08e-43a1-b430-beb62fbf73b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.330107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 774/1542] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:31:41.341554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 769/1543] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.343322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae41199-70c3-4f21-a1e0-cb872c533102 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:41.347758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871baf3-a08e-43a1-b430-beb62fbf73b1 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:41.362278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae41199-70c3-4f21-a1e0-cb872c533102 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:41.364362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871baf3-a08e-43a1-b430-beb62fbf73b1 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:41.385059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a679a64-8c7e-413b-b02f-fe6f488d34d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:41.385059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a679a64-8c7e-413b-b02f-fe6f488d34d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.385059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a679a64-8c7e-413b-b02f-fe6f488d34d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.385059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a679a64-8c7e-413b-b02f-fe6f488d34d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.385549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a679a64-8c7e-413b-b02f-fe6f488d34d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:31:41.385549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a679a64-8c7e-413b-b02f-fe6f488d34d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:41.385715 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c6c9c-0e1c-4086-ae70-0e84c70a05e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:41.385814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: WARNING py.warnings [None req-9a679a64-8c7e-413b-b02f-fe6f488d34d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:31:41.385814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: warnings.warn(msg) Aug 19 16:31:41.385814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: Aug 19 16:31:41.386049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c6c9c-0e1c-4086-ae70-0e84c70a05e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.386049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c6c9c-0e1c-4086-ae70-0e84c70a05e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.386049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c6c9c-0e1c-4086-ae70-0e84c70a05e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.386049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c6c9c-0e1c-4086-ae70-0e84c70a05e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:31:41.386049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c6c9c-0e1c-4086-ae70-0e84c70a05e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:41.388354 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a679a64-8c7e-413b-b02f-fe6f488d34d4 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:41.390604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c6c9c-0e1c-4086-ae70-0e84c70a05e0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:41.417042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 770/1544] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:31:41.425387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 775/1545] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:31:41.426497 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a679a64-8c7e-413b-b02f-fe6f488d34d4 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:41.443398 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a679a64-8c7e-413b-b02f-fe6f488d34d4 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:41.447309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c6c9c-0e1c-4086-ae70-0e84c70a05e0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:41.451133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadee3a-53bb-47e1-b279-d5b430ad0ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:41.451700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadee3a-53bb-47e1-b279-d5b430ad0ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.452089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadee3a-53bb-47e1-b279-d5b430ad0ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.452369 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadee3a-53bb-47e1-b279-d5b430ad0ee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.453031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadee3a-53bb-47e1-b279-d5b430ad0ee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:41.454982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadee3a-53bb-47e1-b279-d5b430ad0ee8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:41.472172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c6c9c-0e1c-4086-ae70-0e84c70a05e0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:41.481280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b92a7-3ca6-4e83-8b11-07a3aef15ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:41.481904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b92a7-3ca6-4e83-8b11-07a3aef15ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.482183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b92a7-3ca6-4e83-8b11-07a3aef15ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.482457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b92a7-3ca6-4e83-8b11-07a3aef15ae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.483086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b92a7-3ca6-4e83-8b11-07a3aef15ae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:41.484117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-8c3b92a7-3ca6-4e83-8b11-07a3aef15ae9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:31:41.484117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Aug 19 16:31:41.484117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: Aug 19 16:31:41.489708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b92a7-3ca6-4e83-8b11-07a3aef15ae9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:41.535095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 771/1546] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:41.540222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadee3a-53bb-47e1-b279-d5b430ad0ee8 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:41.542834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 776/1547] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:41.556272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b92a7-3ca6-4e83-8b11-07a3aef15ae9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:41.642552 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadee3a-53bb-47e1-b279-d5b430ad0ee8 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:41.681305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b92a7-3ca6-4e83-8b11-07a3aef15ae9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:41.695425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91431054-5d31-4d1e-a7a8-03bee2fce239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:41.696060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91431054-5d31-4d1e-a7a8-03bee2fce239 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.696346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91431054-5d31-4d1e-a7a8-03bee2fce239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.696625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91431054-5d31-4d1e-a7a8-03bee2fce239 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.697837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91431054-5d31-4d1e-a7a8-03bee2fce239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:41.700478 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91431054-5d31-4d1e-a7a8-03bee2fce239 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:41.720460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41078c-2b15-4297-855a-dbfff088fb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:41.721095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41078c-2b15-4297-855a-dbfff088fb87 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.721405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41078c-2b15-4297-855a-dbfff088fb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.723163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41078c-2b15-4297-855a-dbfff088fb87 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.724386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41078c-2b15-4297-855a-dbfff088fb87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:41.725874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: WARNING py.warnings [None req-de41078c-2b15-4297-855a-dbfff088fb87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:31:41.725874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: warnings.warn(msg) Aug 19 16:31:41.725874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: Aug 19 16:31:41.727196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41078c-2b15-4297-855a-dbfff088fb87 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:41.729456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 777/1548] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:31:41.730694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e064a6-c5ee-42c7-b47d-0f382d6b15a2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.731074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e064a6-c5ee-42c7-b47d-0f382d6b15a2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.731716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e064a6-c5ee-42c7-b47d-0f382d6b15a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.761022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 772/1549] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:41.761022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b8fcb-be4d-4371-97c7-794e5365a08d None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.761022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b8fcb-be4d-4371-97c7-794e5365a08d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.761022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b8fcb-be4d-4371-97c7-794e5365a08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.763773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-10e064a6-c5ee-42c7-b47d-0f382d6b15a2 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.765116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-10e064a6-c5ee-42c7-b47d-0f382d6b15a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.773301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 778/1550] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.774504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f628-4589-4f0e-90cb-aaa08dfe8369 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.774794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f628-4589-4f0e-90cb-aaa08dfe8369 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.775074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f628-4589-4f0e-90cb-aaa08dfe8369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.781028 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-c41b8fcb-be4d-4371-97c7-794e5365a08d None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.782549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c41b8fcb-be4d-4371-97c7-794e5365a08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.790664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 773/1551] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.791696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54fc4-5278-4869-be49-662e3078f82d None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.791978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54fc4-5278-4869-be49-662e3078f82d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.792247 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54fc4-5278-4869-be49-662e3078f82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.795604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0496f628-4589-4f0e-90cb-aaa08dfe8369 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.797094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0496f628-4589-4f0e-90cb-aaa08dfe8369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.805183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 779/1552] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.806390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c39ac1-d6a9-49a6-9a6e-fd74a0a78ef1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.806675 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c39ac1-d6a9-49a6-9a6e-fd74a0a78ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.806948 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c39ac1-d6a9-49a6-9a6e-fd74a0a78ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.814858 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ffa54fc4-5278-4869-be49-662e3078f82d None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.816068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ffa54fc4-5278-4869-be49-662e3078f82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.824078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 774/1553] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.825547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54fc4-5278-4869-be49-662e3078f82d None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:41.829889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-80c39ac1-d6a9-49a6-9a6e-fd74a0a78ef1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.831119 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-80c39ac1-d6a9-49a6-9a6e-fd74a0a78ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.836825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54fc4-5278-4869-be49-662e3078f82d None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:41.839616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 780/1554] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.840867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c39ac1-d6a9-49a6-9a6e-fd74a0a78ef1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:41.845826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b556d-7865-4690-82c5-1f66a3b08e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:41.846440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b556d-7865-4690-82c5-1f66a3b08e89 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.847358 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b556d-7865-4690-82c5-1f66a3b08e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.847558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b556d-7865-4690-82c5-1f66a3b08e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.848245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b556d-7865-4690-82c5-1f66a3b08e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:41.849875 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b556d-7865-4690-82c5-1f66a3b08e89 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:41.850312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c39ac1-d6a9-49a6-9a6e-fd74a0a78ef1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:41.855501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba7fc4-fc8e-42cd-944f-f7ddb4d42923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:41.855744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba7fc4-fc8e-42cd-944f-f7ddb4d42923 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.856096 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba7fc4-fc8e-42cd-944f-f7ddb4d42923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.856887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba7fc4-fc8e-42cd-944f-f7ddb4d42923 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.857531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba7fc4-fc8e-42cd-944f-f7ddb4d42923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:41.859130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba7fc4-fc8e-42cd-944f-f7ddb4d42923 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:41.862953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 775/1555] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:41.862953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a916792-dc1a-4cfb-bbde-4e5643120954 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.862953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a916792-dc1a-4cfb-bbde-4e5643120954 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.863257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a916792-dc1a-4cfb-bbde-4e5643120954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.872501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 781/1556] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:41.873905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383d886-0641-4c28-954c-20a4121be77a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.874196 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383d886-0641-4c28-954c-20a4121be77a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.874470 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383d886-0641-4c28-954c-20a4121be77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.885715 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-7a916792-dc1a-4cfb-bbde-4e5643120954 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.886921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7a916792-dc1a-4cfb-bbde-4e5643120954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.892851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c383d886-0641-4c28-954c-20a4121be77a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.894191 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c383d886-0641-4c28-954c-20a4121be77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.895273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 776/1557] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.896448 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bfb91-7eac-4062-9c33-4dcdc50e2f64 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.896783 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bfb91-7eac-4062-9c33-4dcdc50e2f64 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.897026 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bfb91-7eac-4062-9c33-4dcdc50e2f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.902750 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 782/1558] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.904886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71eba54-6751-4c25-a83e-77bfd9591546 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.905172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71eba54-6751-4c25-a83e-77bfd9591546 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.905446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71eba54-6751-4c25-a83e-77bfd9591546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.919608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-288bfb91-7eac-4062-9c33-4dcdc50e2f64 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.920548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-288bfb91-7eac-4062-9c33-4dcdc50e2f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.925013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c71eba54-6751-4c25-a83e-77bfd9591546 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.926188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c71eba54-6751-4c25-a83e-77bfd9591546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.928618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 777/1559] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.929821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695fe43-7c03-4b25-b50f-bf292e154074 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.930101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695fe43-7c03-4b25-b50f-bf292e154074 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.930371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695fe43-7c03-4b25-b50f-bf292e154074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.937854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 783/1560] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.937981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:41.939391 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738887e-796c-4550-8e43-5476ba8b1509 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.939622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738887e-796c-4550-8e43-5476ba8b1509 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.939864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738887e-796c-4550-8e43-5476ba8b1509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.954218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f695fe43-7c03-4b25-b50f-bf292e154074 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.955418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f695fe43-7c03-4b25-b50f-bf292e154074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.959977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9738887e-796c-4550-8e43-5476ba8b1509 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:41.961181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9738887e-796c-4550-8e43-5476ba8b1509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:41.963550 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 778/1561] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.964312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695fe43-7c03-4b25-b50f-bf292e154074 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:41.973571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 784/1562] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:41.974335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738887e-796c-4550-8e43-5476ba8b1509 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:41.974470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695fe43-7c03-4b25-b50f-bf292e154074 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:41.981181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad91159-94a2-4155-aaba-8697d27d449b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:41.981772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91159-94a2-4155-aaba-8697d27d449b admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.982046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91159-94a2-4155-aaba-8697d27d449b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.982313 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91159-94a2-4155-aaba-8697d27d449b admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.982945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad91159-94a2-4155-aaba-8697d27d449b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:41.984331 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738887e-796c-4550-8e43-5476ba8b1509 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:41.984521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad91159-94a2-4155-aaba-8697d27d449b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:41.988980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00aabac-2899-4d17-bf83-5047fe332347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:41.989573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00aabac-2899-4d17-bf83-5047fe332347 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.990190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00aabac-2899-4d17-bf83-5047fe332347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.990190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00aabac-2899-4d17-bf83-5047fe332347 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:41.990735 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00aabac-2899-4d17-bf83-5047fe332347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:41.992313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00aabac-2899-4d17-bf83-5047fe332347 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:41.994821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 779/1563] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:41.995805 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2681f-6b18-4043-ac4a-28cd91ed0f67 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:41.996087 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2681f-6b18-4043-ac4a-28cd91ed0f67 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:41.996359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2681f-6b18-4043-ac4a-28cd91ed0f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.006439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 785/1564] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:42.006439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00aabac-2899-4d17-bf83-5047fe332347 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.014085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00aabac-2899-4d17-bf83-5047fe332347 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.018691 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a894-07cf-4ba6-b19c-a87fe1cb4c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.019257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a894-07cf-4ba6-b19c-a87fe1cb4c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.019494 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-8ea2681f-6b18-4043-ac4a-28cd91ed0f67 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:42.019621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a894-07cf-4ba6-b19c-a87fe1cb4c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.019788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a894-07cf-4ba6-b19c-a87fe1cb4c86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.020420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800a894-07cf-4ba6-b19c-a87fe1cb4c86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:31:42.020681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8ea2681f-6b18-4043-ac4a-28cd91ed0f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:42.020825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800a894-07cf-4ba6-b19c-a87fe1cb4c86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.022315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800a894-07cf-4ba6-b19c-a87fe1cb4c86 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.029383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 780/1565] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:42.029956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2681f-6b18-4043-ac4a-28cd91ed0f67 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.038417 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 786/1566] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:31:42.038975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2681f-6b18-4043-ac4a-28cd91ed0f67 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.039691 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a894-07cf-4ba6-b19c-a87fe1cb4c86 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.049660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a894-07cf-4ba6-b19c-a87fe1cb4c86 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.050536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e74aa-614c-488f-8d22-b4c0989acad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.051144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e74aa-614c-488f-8d22-b4c0989acad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.051425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e74aa-614c-488f-8d22-b4c0989acad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.051697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e74aa-614c-488f-8d22-b4c0989acad9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.052341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e74aa-614c-488f-8d22-b4c0989acad9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:31:42.052645 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e74aa-614c-488f-8d22-b4c0989acad9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.054282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e74aa-614c-488f-8d22-b4c0989acad9 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.055228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30716393-58e2-41ed-bbf3-cf4ef5bfec89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.055812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30716393-58e2-41ed-bbf3-cf4ef5bfec89 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.056090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30716393-58e2-41ed-bbf3-cf4ef5bfec89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.056360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30716393-58e2-41ed-bbf3-cf4ef5bfec89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.057017 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30716393-58e2-41ed-bbf3-cf4ef5bfec89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:31:42.057314 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30716393-58e2-41ed-bbf3-cf4ef5bfec89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.058875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30716393-58e2-41ed-bbf3-cf4ef5bfec89 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.066113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 781/1567] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:31:42.066909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e74aa-614c-488f-8d22-b4c0989acad9 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.070529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 787/1568] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:31:42.073658 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30716393-58e2-41ed-bbf3-cf4ef5bfec89 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.079018 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e74aa-614c-488f-8d22-b4c0989acad9 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.083841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b7337-f667-4544-8093-e03fbc85f2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.084251 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30716393-58e2-41ed-bbf3-cf4ef5bfec89 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.084428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b7337-f667-4544-8093-e03fbc85f2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.084704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b7337-f667-4544-8093-e03fbc85f2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.085010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b7337-f667-4544-8093-e03fbc85f2d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.085642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b7337-f667-4544-8093-e03fbc85f2d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:31:42.085934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b7337-f667-4544-8093-e03fbc85f2d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.087766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b7337-f667-4544-8093-e03fbc85f2d9 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.089127 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df017a32-a44c-468b-86d2-60de65c41e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.089715 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df017a32-a44c-468b-86d2-60de65c41e69 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.089998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df017a32-a44c-468b-86d2-60de65c41e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.090295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df017a32-a44c-468b-86d2-60de65c41e69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.091475 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df017a32-a44c-468b-86d2-60de65c41e69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:31:42.091926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df017a32-a44c-468b-86d2-60de65c41e69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.093528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df017a32-a44c-468b-86d2-60de65c41e69 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.100529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 782/1569] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:31:42.114318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:42.114426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b7337-f667-4544-8093-e03fbc85f2d9 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.114520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 788/1570] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:31:42.114520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df017a32-a44c-468b-86d2-60de65c41e69 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.115730 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b7337-f667-4544-8093-e03fbc85f2d9 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.120517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180176b-c08f-48f9-9246-2b2fb72a7fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.121195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180176b-c08f-48f9-9246-2b2fb72a7fea admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.121473 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180176b-c08f-48f9-9246-2b2fb72a7fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.122067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180176b-c08f-48f9-9246-2b2fb72a7fea admin admin] PATH_INFO: `/v3/projects/e50130a56a8a4cd5be25f7350d22cf05/users/e2ada95813374bdf894a1ffcbbe1c7da/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.123851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df017a32-a44c-468b-86d2-60de65c41e69 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.128554 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c042e0f-0e6e-43d8-80e6-6b47a80ad631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.129345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c042e0f-0e6e-43d8-80e6-6b47a80ad631 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.129629 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c042e0f-0e6e-43d8-80e6-6b47a80ad631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.129900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c042e0f-0e6e-43d8-80e6-6b47a80ad631 admin admin] PATH_INFO: `/v3/projects/1743538991c8415394ac6cd5aed28d6a/users/9b0455ae3ec94c929c940df6a0f6687d/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.138939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180176b-c08f-48f9-9246-2b2fb72a7fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50130a56a8a4cd5be25f7350d22cf05, user_id=e2ada95813374bdf894a1ffcbbe1c7da, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.141743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180176b-c08f-48f9-9246-2b2fb72a7fea admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.145945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c042e0f-0e6e-43d8-80e6-6b47a80ad631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1743538991c8415394ac6cd5aed28d6a, user_id=9b0455ae3ec94c929c940df6a0f6687d, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.147292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-3c042e0f-0e6e-43d8-80e6-6b47a80ad631 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:31:42.147292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Aug 19 16:31:42.147292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: Aug 19 16:31:42.149881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c042e0f-0e6e-43d8-80e6-6b47a80ad631 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.156954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 783/1571] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:42 2022] PUT /identity/v3/projects/e50130a56a8a4cd5be25f7350d22cf05/users/e2ada95813374bdf894a1ffcbbe1c7da/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:42.157670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180176b-c08f-48f9-9246-2b2fb72a7fea admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.163776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 789/1572] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:42 2022] PUT /identity/v3/projects/1743538991c8415394ac6cd5aed28d6a/users/9b0455ae3ec94c929c940df6a0f6687d/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:42.164555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c042e0f-0e6e-43d8-80e6-6b47a80ad631 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.226903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c042e0f-0e6e-43d8-80e6-6b47a80ad631 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.227985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180176b-c08f-48f9-9246-2b2fb72a7fea admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.231687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d2a51-039f-43ff-a2b6-c0c40a4b2947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.232290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d2a51-039f-43ff-a2b6-c0c40a4b2947 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.232570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d2a51-039f-43ff-a2b6-c0c40a4b2947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.232872 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d2a51-039f-43ff-a2b6-c0c40a4b2947 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.232994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86f2ec-6bff-451e-a727-b27771a26d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.233491 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d2a51-039f-43ff-a2b6-c0c40a4b2947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.233607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86f2ec-6bff-451e-a727-b27771a26d60 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.233804 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86f2ec-6bff-451e-a727-b27771a26d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.234070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86f2ec-6bff-451e-a727-b27771a26d60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.234701 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86f2ec-6bff-451e-a727-b27771a26d60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:31:42.234992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86f2ec-6bff-451e-a727-b27771a26d60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.235442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d2a51-039f-43ff-a2b6-c0c40a4b2947 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.236548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86f2ec-6bff-451e-a727-b27771a26d60 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.255156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 784/1573] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:31:42.255156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86f2ec-6bff-451e-a727-b27771a26d60 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.259126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86f2ec-6bff-451e-a727-b27771a26d60 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.262990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 790/1574] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/projects => generated 357 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:42.263795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d2a51-039f-43ff-a2b6-c0c40a4b2947 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.264553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba08026-f4c4-4751-9b48-5365025c8d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.265218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba08026-f4c4-4751-9b48-5365025c8d9f admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.265495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba08026-f4c4-4751-9b48-5365025c8d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.265759 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba08026-f4c4-4751-9b48-5365025c8d9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.266381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba08026-f4c4-4751-9b48-5365025c8d9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.268296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba08026-f4c4-4751-9b48-5365025c8d9f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.296289 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 785/1575] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:42.296909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba08026-f4c4-4751-9b48-5365025c8d9f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.328405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d2a51-039f-43ff-a2b6-c0c40a4b2947 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.360625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba08026-f4c4-4751-9b48-5365025c8d9f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.366018 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdeb1da-72fb-42a5-a34a-4febdcaa8b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.366621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdeb1da-72fb-42a5-a34a-4febdcaa8b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.366896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdeb1da-72fb-42a5-a34a-4febdcaa8b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.367164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdeb1da-72fb-42a5-a34a-4febdcaa8b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.367786 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdeb1da-72fb-42a5-a34a-4febdcaa8b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.369712 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdeb1da-72fb-42a5-a34a-4febdcaa8b4a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.374983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:42.384658 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da44f70-4f31-4339-b711-6997566f8f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.385533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da44f70-4f31-4339-b711-6997566f8f58 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.385836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da44f70-4f31-4339-b711-6997566f8f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.386107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da44f70-4f31-4339-b711-6997566f8f58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.386732 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da44f70-4f31-4339-b711-6997566f8f58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.389369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da44f70-4f31-4339-b711-6997566f8f58 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.399528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 786/1576] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/projects => generated 393 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:42.400339 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdeb1da-72fb-42a5-a34a-4febdcaa8b4a admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.405977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:42.423193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 791/1577] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:42.423193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23426-11c3-4a91-bfab-493898e3197a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.423193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23426-11c3-4a91-bfab-493898e3197a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.423193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23426-11c3-4a91-bfab-493898e3197a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.440674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-cea23426-11c3-4a91-bfab-493898e3197a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:42.441935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cea23426-11c3-4a91-bfab-493898e3197a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:42.463097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdeb1da-72fb-42a5-a34a-4febdcaa8b4a admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.500817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 792/1578] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:42.504252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14985b0e-9d5c-4c3e-8ad4-7e14ddaa41ec None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.504252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14985b0e-9d5c-4c3e-8ad4-7e14ddaa41ec None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.504252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14985b0e-9d5c-4c3e-8ad4-7e14ddaa41ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.524658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf94b9-0f02-4eaf-8fe9-c62e11daed0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.524931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf94b9-0f02-4eaf-8fe9-c62e11daed0f admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.525130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf94b9-0f02-4eaf-8fe9-c62e11daed0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.525398 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf94b9-0f02-4eaf-8fe9-c62e11daed0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.526017 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf94b9-0f02-4eaf-8fe9-c62e11daed0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.527977 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf94b9-0f02-4eaf-8fe9-c62e11daed0f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.531713 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-14985b0e-9d5c-4c3e-8ad4-7e14ddaa41ec None None] MFA Rules not processed for user `e2ada95813374bdf894a1ffcbbe1c7da`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:42.532966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-14985b0e-9d5c-4c3e-8ad4-7e14ddaa41ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:42.558347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 787/1579] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:42.558347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf94b9-0f02-4eaf-8fe9-c62e11daed0f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.601201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 793/1580] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:42.602404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14985b0e-9d5c-4c3e-8ad4-7e14ddaa41ec None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.615693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14985b0e-9d5c-4c3e-8ad4-7e14ddaa41ec None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.620187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ecce7-e67b-4bfe-a7d3-6c80d707fee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.620799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ecce7-e67b-4bfe-a7d3-6c80d707fee9 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.621084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ecce7-e67b-4bfe-a7d3-6c80d707fee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.621354 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ecce7-e67b-4bfe-a7d3-6c80d707fee9 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.622646 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ecce7-e67b-4bfe-a7d3-6c80d707fee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.622893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf94b9-0f02-4eaf-8fe9-c62e11daed0f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.624571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ecce7-e67b-4bfe-a7d3-6c80d707fee9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.627614 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bd590-b438-417a-8726-cc99d3524712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.628905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bd590-b438-417a-8726-cc99d3524712 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.628905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bd590-b438-417a-8726-cc99d3524712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.628905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bd590-b438-417a-8726-cc99d3524712 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.629437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bd590-b438-417a-8726-cc99d3524712 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.631325 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bd590-b438-417a-8726-cc99d3524712 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.649157 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 794/1581] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/users => generated 279 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:42.650025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ecce7-e67b-4bfe-a7d3-6c80d707fee9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.661094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ecce7-e67b-4bfe-a7d3-6c80d707fee9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.665845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 788/1582] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:42.666238 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df2b6e-3cd3-47fe-b34a-5768355b78f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.666861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df2b6e-3cd3-47fe-b34a-5768355b78f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.666987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bd590-b438-417a-8726-cc99d3524712 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.667143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df2b6e-3cd3-47fe-b34a-5768355b78f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.667414 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df2b6e-3cd3-47fe-b34a-5768355b78f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.668045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df2b6e-3cd3-47fe-b34a-5768355b78f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.670665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df2b6e-3cd3-47fe-b34a-5768355b78f3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.673539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:42.693315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 795/1583] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:42.694258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df2b6e-3cd3-47fe-b34a-5768355b78f3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.704486 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df2b6e-3cd3-47fe-b34a-5768355b78f3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.738178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bd590-b438-417a-8726-cc99d3524712 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.743020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0af29c-895c-44c4-80f5-ee37d17e05e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.743634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0af29c-895c-44c4-80f5-ee37d17e05e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.743920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0af29c-895c-44c4-80f5-ee37d17e05e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.744199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0af29c-895c-44c4-80f5-ee37d17e05e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.745255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0af29c-895c-44c4-80f5-ee37d17e05e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.752581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:42.753401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0af29c-895c-44c4-80f5-ee37d17e05e8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.769588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7390011-ae5e-46b8-ba8c-1d5e832ace6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.770236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7390011-ae5e-46b8-ba8c-1d5e832ace6d admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.770551 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7390011-ae5e-46b8-ba8c-1d5e832ace6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.770911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7390011-ae5e-46b8-ba8c-1d5e832ace6d admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.771583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7390011-ae5e-46b8-ba8c-1d5e832ace6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.773565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7390011-ae5e-46b8-ba8c-1d5e832ace6d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.775797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 789/1584] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:42.776821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0af29c-895c-44c4-80f5-ee37d17e05e8 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.799757 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 796/1585] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:42.799757 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7390011-ae5e-46b8-ba8c-1d5e832ace6d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.807148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7390011-ae5e-46b8-ba8c-1d5e832ace6d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.812901 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558aeb95-4c00-46b5-b402-a5bdfd98018c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.813585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aeb95-4c00-46b5-b402-a5bdfd98018c admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.813868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aeb95-4c00-46b5-b402-a5bdfd98018c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.814197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aeb95-4c00-46b5-b402-a5bdfd98018c admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.814850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558aeb95-4c00-46b5-b402-a5bdfd98018c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.816954 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558aeb95-4c00-46b5-b402-a5bdfd98018c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.838319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0af29c-895c-44c4-80f5-ee37d17e05e8 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.839892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 797/1586] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/users => generated 283 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:42.840636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558aeb95-4c00-46b5-b402-a5bdfd98018c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.844530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c8c0a-5a3d-4050-b8d7-db825551dc7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.845174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c8c0a-5a3d-4050-b8d7-db825551dc7d service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.845631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c8c0a-5a3d-4050-b8d7-db825551dc7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.845751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c8c0a-5a3d-4050-b8d7-db825551dc7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.854888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c8c0a-5a3d-4050-b8d7-db825551dc7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.857044 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c8c0a-5a3d-4050-b8d7-db825551dc7d service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.866400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 790/1587] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:42.867176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c8c0a-5a3d-4050-b8d7-db825551dc7d service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.876742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c8c0a-5a3d-4050-b8d7-db825551dc7d service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.881545 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117518d4-b47b-4506-8d4c-56feda23373e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.882138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117518d4-b47b-4506-8d4c-56feda23373e admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.882418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117518d4-b47b-4506-8d4c-56feda23373e admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.882689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117518d4-b47b-4506-8d4c-56feda23373e admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.883317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117518d4-b47b-4506-8d4c-56feda23373e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.885056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117518d4-b47b-4506-8d4c-56feda23373e admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.897531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 791/1588] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:42.899140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117518d4-b47b-4506-8d4c-56feda23373e admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.904970 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558aeb95-4c00-46b5-b402-a5bdfd98018c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.908715 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfc07c-7e57-4ce3-bc1d-eb86458fa4be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.909398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfc07c-7e57-4ce3-bc1d-eb86458fa4be service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.909541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfc07c-7e57-4ce3-bc1d-eb86458fa4be service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.909812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfc07c-7e57-4ce3-bc1d-eb86458fa4be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.910918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117518d4-b47b-4506-8d4c-56feda23373e admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.916115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340de36-787e-4529-81b0-bab151b56dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.916793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340de36-787e-4529-81b0-bab151b56dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.917076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340de36-787e-4529-81b0-bab151b56dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.917351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340de36-787e-4529-81b0-bab151b56dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.917974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340de36-787e-4529-81b0-bab151b56dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.920042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340de36-787e-4529-81b0-bab151b56dc5 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.947261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 792/1589] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:31:42 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:42.947261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340de36-787e-4529-81b0-bab151b56dc5 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.956490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340de36-787e-4529-81b0-bab151b56dc5 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.962547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714fc41-148a-4fe5-a52c-6db39b50c43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.963210 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714fc41-148a-4fe5-a52c-6db39b50c43b admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.963601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714fc41-148a-4fe5-a52c-6db39b50c43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.963888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714fc41-148a-4fe5-a52c-6db39b50c43b admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.964775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714fc41-148a-4fe5-a52c-6db39b50c43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.967395 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714fc41-148a-4fe5-a52c-6db39b50c43b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.979737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 793/1590] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:42.980722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714fc41-148a-4fe5-a52c-6db39b50c43b admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.985316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfc07c-7e57-4ce3-bc1d-eb86458fa4be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:42.986885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfc07c-7e57-4ce3-bc1d-eb86458fa4be service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:42.992089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714fc41-148a-4fe5-a52c-6db39b50c43b admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:42.997620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 798/1591] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:42.997845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df693c64-562b-4933-a3d6-47a7ee5999ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:42.998506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df693c64-562b-4933-a3d6-47a7ee5999ef admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:42.998930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df693c64-562b-4933-a3d6-47a7ee5999ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:42.999206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df693c64-562b-4933-a3d6-47a7ee5999ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:42.999479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfc07c-7e57-4ce3-bc1d-eb86458fa4be service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:42.999894 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df693c64-562b-4933-a3d6-47a7ee5999ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.001717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df693c64-562b-4933-a3d6-47a7ee5999ef admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.012818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfc07c-7e57-4ce3-bc1d-eb86458fa4be service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.017780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f72036-97f1-40f4-ae13-a4f0636a1ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.018401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f72036-97f1-40f4-ae13-a4f0636a1ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.018683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f72036-97f1-40f4-ae13-a4f0636a1ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.018966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f72036-97f1-40f4-ae13-a4f0636a1ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.019644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f72036-97f1-40f4-ae13-a4f0636a1ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.023767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 794/1592] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:43.023767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df693c64-562b-4933-a3d6-47a7ee5999ef admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.023994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f72036-97f1-40f4-ae13-a4f0636a1ef7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.034581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df693c64-562b-4933-a3d6-47a7ee5999ef admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.039592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 799/1593] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:43.040295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f72036-97f1-40f4-ae13-a4f0636a1ef7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.040572 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6257c-af43-4006-9308-8291fc2ec097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.041318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6257c-af43-4006-9308-8291fc2ec097 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.041739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6257c-af43-4006-9308-8291fc2ec097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.042113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6257c-af43-4006-9308-8291fc2ec097 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.043176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6257c-af43-4006-9308-8291fc2ec097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.045092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6257c-af43-4006-9308-8291fc2ec097 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.051716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f72036-97f1-40f4-ae13-a4f0636a1ef7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.056100 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 795/1594] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:43.056654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55db0b-052e-47bd-91e8-97f01fedc293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.057302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55db0b-052e-47bd-91e8-97f01fedc293 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.057592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55db0b-052e-47bd-91e8-97f01fedc293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.057917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55db0b-052e-47bd-91e8-97f01fedc293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.058468 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55db0b-052e-47bd-91e8-97f01fedc293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.061159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55db0b-052e-47bd-91e8-97f01fedc293 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.061401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6257c-af43-4006-9308-8291fc2ec097 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.077630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6257c-af43-4006-9308-8291fc2ec097 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.085922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9b75-3723-47f6-9a6c-b116793cf802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.087126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9b75-3723-47f6-9a6c-b116793cf802 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.087693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9b75-3723-47f6-9a6c-b116793cf802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.088179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9b75-3723-47f6-9a6c-b116793cf802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.089206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b9b75-3723-47f6-9a6c-b116793cf802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.091470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b9b75-3723-47f6-9a6c-b116793cf802 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.099527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 800/1595] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:43.100649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55db0b-052e-47bd-91e8-97f01fedc293 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.108510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 796/1596] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:43.109231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9b75-3723-47f6-9a6c-b116793cf802 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.117239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55db0b-052e-47bd-91e8-97f01fedc293 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.122227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9b75-3723-47f6-9a6c-b116793cf802 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.124929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308a3f9-b4bd-417c-917b-686e620bb680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.125562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308a3f9-b4bd-417c-917b-686e620bb680 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.125839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308a3f9-b4bd-417c-917b-686e620bb680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.126108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308a3f9-b4bd-417c-917b-686e620bb680 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.127120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308a3f9-b4bd-417c-917b-686e620bb680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.128492 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed114149-1256-434e-84bf-a0b5b33433c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.128978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308a3f9-b4bd-417c-917b-686e620bb680 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.129360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114149-1256-434e-84bf-a0b5b33433c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.129784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114149-1256-434e-84bf-a0b5b33433c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.130156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114149-1256-434e-84bf-a0b5b33433c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.130977 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed114149-1256-434e-84bf-a0b5b33433c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.132788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed114149-1256-434e-84bf-a0b5b33433c3 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.148178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 801/1597] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:43.148178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308a3f9-b4bd-417c-917b-686e620bb680 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.148504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 797/1598] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:43.148504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed114149-1256-434e-84bf-a0b5b33433c3 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.161327 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed114149-1256-434e-84bf-a0b5b33433c3 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.165541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308a3f9-b4bd-417c-917b-686e620bb680 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.167035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69681b59-c102-420f-943c-e740bf80d827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.168141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69681b59-c102-420f-943c-e740bf80d827 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.168658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69681b59-c102-420f-943c-e740bf80d827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.169216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69681b59-c102-420f-943c-e740bf80d827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.170113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69681b59-c102-420f-943c-e740bf80d827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.172335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69681b59-c102-420f-943c-e740bf80d827 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.174532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11f4d0-c256-4b92-826a-daab8c1d21fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.175164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11f4d0-c256-4b92-826a-daab8c1d21fc admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.175922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11f4d0-c256-4b92-826a-daab8c1d21fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.176236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11f4d0-c256-4b92-826a-daab8c1d21fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.177160 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11f4d0-c256-4b92-826a-daab8c1d21fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.179654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11f4d0-c256-4b92-826a-daab8c1d21fc admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.180194 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:43.190464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 798/1599] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:43.191655 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69681b59-c102-420f-943c-e740bf80d827 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.198801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 802/1600] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:43.200469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11f4d0-c256-4b92-826a-daab8c1d21fc admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.213463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69681b59-c102-420f-943c-e740bf80d827 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.216330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11f4d0-c256-4b92-826a-daab8c1d21fc admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.220457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364de99-1cad-441a-a9a9-0f1ad69a3b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.221584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364de99-1cad-441a-a9a9-0f1ad69a3b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.221584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364de99-1cad-441a-a9a9-0f1ad69a3b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.221806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364de99-1cad-441a-a9a9-0f1ad69a3b09 admin admin] PATH_INFO: `/v3/projects/e0c2404dc80e4cf9ad6b109b3aabb9d5/users/21be6a5796114b218cd028f57a437cdd/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.224382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b9d3b-37cf-4054-8bb6-5e8b6a1d6417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.225837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b9d3b-37cf-4054-8bb6-5e8b6a1d6417 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.227266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b9d3b-37cf-4054-8bb6-5e8b6a1d6417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.228321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b9d3b-37cf-4054-8bb6-5e8b6a1d6417 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.229437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b9d3b-37cf-4054-8bb6-5e8b6a1d6417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.231784 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b9d3b-37cf-4054-8bb6-5e8b6a1d6417 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.242970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364de99-1cad-441a-a9a9-0f1ad69a3b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0c2404dc80e4cf9ad6b109b3aabb9d5, user_id=21be6a5796114b218cd028f57a437cdd, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.246191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364de99-1cad-441a-a9a9-0f1ad69a3b09 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.252171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 803/1601] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:43.252171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b9d3b-37cf-4054-8bb6-5e8b6a1d6417 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.268149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b9d3b-37cf-4054-8bb6-5e8b6a1d6417 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.272065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 799/1602] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:43 2022] PUT /identity/v3/projects/e0c2404dc80e4cf9ad6b109b3aabb9d5/users/21be6a5796114b218cd028f57a437cdd/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:43.273158 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364de99-1cad-441a-a9a9-0f1ad69a3b09 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.304345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:43.370223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d4c66-991e-463a-aa41-42a54f9e8b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.371079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d4c66-991e-463a-aa41-42a54f9e8b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.371492 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d4c66-991e-463a-aa41-42a54f9e8b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.371971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d4c66-991e-463a-aa41-42a54f9e8b9a admin admin] PATH_INFO: `/v3/projects/b4516d3cd63d4d89824c4a0b20b6eea1/users/8d20b306fc68435baa0da6f09c915157/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.393690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364de99-1cad-441a-a9a9-0f1ad69a3b09 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.396627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d4c66-991e-463a-aa41-42a54f9e8b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4516d3cd63d4d89824c4a0b20b6eea1, user_id=8d20b306fc68435baa0da6f09c915157, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.400650 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf7a5b-7fa8-4eb4-b3c9-de489960cd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.400868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d4c66-991e-463a-aa41-42a54f9e8b9a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.401692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf7a5b-7fa8-4eb4-b3c9-de489960cd2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.402009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf7a5b-7fa8-4eb4-b3c9-de489960cd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.402479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf7a5b-7fa8-4eb4-b3c9-de489960cd2d admin admin] PATH_INFO: `/v3/projects/b5638f78ab9440479a209d0023e6d0bd/users/7936bf23a031418b8ce304aa3bbcf8a2/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.422937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf7a5b-7fa8-4eb4-b3c9-de489960cd2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5638f78ab9440479a209d0023e6d0bd, user_id=7936bf23a031418b8ce304aa3bbcf8a2, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.423931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 804/1603] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:43 2022] PUT /identity/v3/projects/b4516d3cd63d4d89824c4a0b20b6eea1/users/8d20b306fc68435baa0da6f09c915157/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:43.426148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d4c66-991e-463a-aa41-42a54f9e8b9a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.426278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf7a5b-7fa8-4eb4-b3c9-de489960cd2d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.442922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 800/1604] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:43 2022] PUT /identity/v3/projects/b5638f78ab9440479a209d0023e6d0bd/users/7936bf23a031418b8ce304aa3bbcf8a2/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:43.446152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf7a5b-7fa8-4eb4-b3c9-de489960cd2d admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.510313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d4c66-991e-463a-aa41-42a54f9e8b9a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.520165 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b39ab3-b74b-44ad-8735-d3191330e646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.520270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf7a5b-7fa8-4eb4-b3c9-de489960cd2d admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.520361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b39ab3-b74b-44ad-8735-d3191330e646 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.520361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b39ab3-b74b-44ad-8735-d3191330e646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.520361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b39ab3-b74b-44ad-8735-d3191330e646 admin admin] PATH_INFO: `/v3/projects/d92de66a9f7543eb8c917c1a6ddfcd55/users/072cfcc77e9f41e0b0bbe67729b74be8/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.524880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a917c63-d8d1-4344-8aaf-4956ed540ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.525931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a917c63-d8d1-4344-8aaf-4956ed540ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.525931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a917c63-d8d1-4344-8aaf-4956ed540ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.526223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a917c63-d8d1-4344-8aaf-4956ed540ad3 admin admin] PATH_INFO: `/v3/projects/d22d9f4004394e1cbcd0a2afffd114b1/users/e09fd34737af41dc972d806dc3b0705e/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.545013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a917c63-d8d1-4344-8aaf-4956ed540ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22d9f4004394e1cbcd0a2afffd114b1, user_id=e09fd34737af41dc972d806dc3b0705e, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.549190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a917c63-d8d1-4344-8aaf-4956ed540ad3 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.550297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b39ab3-b74b-44ad-8735-d3191330e646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92de66a9f7543eb8c917c1a6ddfcd55, user_id=072cfcc77e9f41e0b0bbe67729b74be8, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.553809 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b39ab3-b74b-44ad-8735-d3191330e646 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.569947 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 801/1605] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:43 2022] PUT /identity/v3/projects/d22d9f4004394e1cbcd0a2afffd114b1/users/e09fd34737af41dc972d806dc3b0705e/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:43.570908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a917c63-d8d1-4344-8aaf-4956ed540ad3 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:43.572348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 805/1606] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:43 2022] PUT /identity/v3/projects/d92de66a9f7543eb8c917c1a6ddfcd55/users/072cfcc77e9f41e0b0bbe67729b74be8/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:43.587310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132d9b8-bc1e-4ed5-a832-108976a07386 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.587765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132d9b8-bc1e-4ed5-a832-108976a07386 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.588531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132d9b8-bc1e-4ed5-a832-108976a07386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.623172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2132d9b8-bc1e-4ed5-a832-108976a07386 None None] MFA Rules not processed for user `21be6a5796114b218cd028f57a437cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:43.624699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2132d9b8-bc1e-4ed5-a832-108976a07386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:43.670336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:43.702924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a917c63-d8d1-4344-8aaf-4956ed540ad3 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:43.782169 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53b8ba-b2ae-4eae-a905-1b155ac05420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:43.782998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53b8ba-b2ae-4eae-a905-1b155ac05420 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.783407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53b8ba-b2ae-4eae-a905-1b155ac05420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.783787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53b8ba-b2ae-4eae-a905-1b155ac05420 admin admin] PATH_INFO: `/v3/projects/e15ca27a2c874997bd01de973a948311/users/d55dc52ab80642da90f05e74092ffdf8/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.803094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53b8ba-b2ae-4eae-a905-1b155ac05420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e15ca27a2c874997bd01de973a948311, user_id=d55dc52ab80642da90f05e74092ffdf8, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:43.806698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53b8ba-b2ae-4eae-a905-1b155ac05420 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:43.813959 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 806/1607] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:43.815702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba27b3-1836-49fb-bf62-80efc63f7b6a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.816196 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba27b3-1836-49fb-bf62-80efc63f7b6a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.816677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba27b3-1836-49fb-bf62-80efc63f7b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.840061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 802/1608] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:43 2022] PUT /identity/v3/projects/e15ca27a2c874997bd01de973a948311/users/d55dc52ab80642da90f05e74092ffdf8/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:43.841646 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-93ba27b3-1836-49fb-bf62-80efc63f7b6a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:43.842174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ee419-e4d8-4479-9437-25c94d062568 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.842664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ee419-e4d8-4479-9437-25c94d062568 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.842874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-93ba27b3-1836-49fb-bf62-80efc63f7b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:43.842997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ee419-e4d8-4479-9437-25c94d062568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.855545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:43.870832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b68ee419-e4d8-4479-9437-25c94d062568 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:43.873375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b68ee419-e4d8-4479-9437-25c94d062568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:43.942157 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 807/1609] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:43.943457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2748c8-7092-4ee2-a3f7-09e1b3492786 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.943746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2748c8-7092-4ee2-a3f7-09e1b3492786 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.944027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2748c8-7092-4ee2-a3f7-09e1b3492786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.967425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 803/1610] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:43.969126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdedc6f-ace6-4ef1-9450-e1277c47a0ef None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:43.969554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdedc6f-ace6-4ef1-9450-e1277c47a0ef None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:43.970154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdedc6f-ace6-4ef1-9450-e1277c47a0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:43.990242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ee2748c8-7092-4ee2-a3f7-09e1b3492786 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:43.991730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ee2748c8-7092-4ee2-a3f7-09e1b3492786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:44.002662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 808/1611] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:44.005416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2748c8-7092-4ee2-a3f7-09e1b3492786 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.011018 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-9cdedc6f-ace6-4ef1-9450-e1277c47a0ef None None] MFA Rules not processed for user `072cfcc77e9f41e0b0bbe67729b74be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:44.013255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9cdedc6f-ace6-4ef1-9450-e1277c47a0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:44.030596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2748c8-7092-4ee2-a3f7-09e1b3492786 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.037304 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d8793-63e7-43ad-a7d9-655ead121093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.037939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d8793-63e7-43ad-a7d9-655ead121093 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.038251 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d8793-63e7-43ad-a7d9-655ead121093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.038527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d8793-63e7-43ad-a7d9-655ead121093 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.039191 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d8793-63e7-43ad-a7d9-655ead121093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.041327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d8793-63e7-43ad-a7d9-655ead121093 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.097558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 809/1612] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:31:44 2022] POST /identity/v3/projects => generated 355 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:44.098833 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1e63f-d8c2-489e-953c-2538ce62ee18 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.099748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1e63f-d8c2-489e-953c-2538ce62ee18 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.100023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1e63f-d8c2-489e-953c-2538ce62ee18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.123292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6ff1e63f-d8c2-489e-953c-2538ce62ee18 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:44.124574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6ff1e63f-d8c2-489e-953c-2538ce62ee18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:44.137196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 804/1613] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:44.137196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdedc6f-ace6-4ef1-9450-e1277c47a0ef None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.138483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9cdedc6f-ace6-4ef1-9450-e1277c47a0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:44.146247 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:44.200834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 810/1614] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:44.200834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1e63f-d8c2-489e-953c-2538ce62ee18 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.200834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6ff1e63f-d8c2-489e-953c-2538ce62ee18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:44.214849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdedc6f-ace6-4ef1-9450-e1277c47a0ef None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.273009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a2bb2-9dc8-47fb-a77f-bc558b19fe46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.273987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a2bb2-9dc8-47fb-a77f-bc558b19fe46 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.274187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a2bb2-9dc8-47fb-a77f-bc558b19fe46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.275188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a2bb2-9dc8-47fb-a77f-bc558b19fe46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.276336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1e63f-d8c2-489e-953c-2538ce62ee18 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.281378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e813bf-2668-41ff-9cd0-5f054893cb8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.282166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e813bf-2668-41ff-9cd0-5f054893cb8c service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.282802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e813bf-2668-41ff-9cd0-5f054893cb8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.283202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e813bf-2668-41ff-9cd0-5f054893cb8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.287680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a2bb2-9dc8-47fb-a77f-bc558b19fe46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.289247 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a2bb2-9dc8-47fb-a77f-bc558b19fe46 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.298465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e813bf-2668-41ff-9cd0-5f054893cb8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.299946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e813bf-2668-41ff-9cd0-5f054893cb8c service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.306424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 805/1615] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:44.307511 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a2bb2-9dc8-47fb-a77f-bc558b19fe46 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.311455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 811/1616] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:44.312294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e813bf-2668-41ff-9cd0-5f054893cb8c service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.319660 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a2bb2-9dc8-47fb-a77f-bc558b19fe46 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.322704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e813bf-2668-41ff-9cd0-5f054893cb8c service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.328341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824810a4-6dc3-4db8-9876-34801ebfb977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.329645 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824810a4-6dc3-4db8-9876-34801ebfb977 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.330482 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60f692-a91b-4217-a545-019175896da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.331419 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824810a4-6dc3-4db8-9876-34801ebfb977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.331638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824810a4-6dc3-4db8-9876-34801ebfb977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.332169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60f692-a91b-4217-a545-019175896da1 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.332445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60f692-a91b-4217-a545-019175896da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.332709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60f692-a91b-4217-a545-019175896da1 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.334954 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca60f692-a91b-4217-a545-019175896da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.338579 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca60f692-a91b-4217-a545-019175896da1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.348593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824810a4-6dc3-4db8-9876-34801ebfb977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.350124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824810a4-6dc3-4db8-9876-34801ebfb977 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.363740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 806/1617] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:44.364621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824810a4-6dc3-4db8-9876-34801ebfb977 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.372956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 812/1618] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:31:44 2022] POST /identity/v3/users => generated 284 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:44.374317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60f692-a91b-4217-a545-019175896da1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.378064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824810a4-6dc3-4db8-9876-34801ebfb977 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.383749 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedc076-e1ff-46f2-ba97-8fc0b42dac17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.384604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedc076-e1ff-46f2-ba97-8fc0b42dac17 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.385073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedc076-e1ff-46f2-ba97-8fc0b42dac17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.385360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedc076-e1ff-46f2-ba97-8fc0b42dac17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.386210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60f692-a91b-4217-a545-019175896da1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.386502 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedc076-e1ff-46f2-ba97-8fc0b42dac17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.390668 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedc076-e1ff-46f2-ba97-8fc0b42dac17 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.390765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0726d-5996-4778-b143-1234d51f8b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.390866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0726d-5996-4778-b143-1234d51f8b2b service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.398216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0726d-5996-4778-b143-1234d51f8b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.398736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0726d-5996-4778-b143-1234d51f8b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.410708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0726d-5996-4778-b143-1234d51f8b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.412212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0726d-5996-4778-b143-1234d51f8b2b service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.419113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:44.425545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 813/1619] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:44.426563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0726d-5996-4778-b143-1234d51f8b2b service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.444620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0726d-5996-4778-b143-1234d51f8b2b service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.452078 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444cedc6-1923-4f09-af79-4178a198d2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.452636 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 807/1620] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:31:44 2022] POST /identity/v3/projects => generated 363 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:44.453338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444cedc6-1923-4f09-af79-4178a198d2fb admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.453918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444cedc6-1923-4f09-af79-4178a198d2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.454243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444cedc6-1923-4f09-af79-4178a198d2fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.454982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444cedc6-1923-4f09-af79-4178a198d2fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.456787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedc076-e1ff-46f2-ba97-8fc0b42dac17 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.456991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444cedc6-1923-4f09-af79-4178a198d2fb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.475755 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 814/1621] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:44.478225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444cedc6-1923-4f09-af79-4178a198d2fb admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.491892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444cedc6-1923-4f09-af79-4178a198d2fb admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.596171 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedc076-e1ff-46f2-ba97-8fc0b42dac17 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.601538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49d79a-37b1-49a4-a662-c14afdb02530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.611247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49d79a-37b1-49a4-a662-c14afdb02530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.611961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49d79a-37b1-49a4-a662-c14afdb02530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.612471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49d79a-37b1-49a4-a662-c14afdb02530 admin admin] PATH_INFO: `/v3/projects/ad847618ee09484a96cda3bd83da358f/users/0418a67fb0114b809b01ee6fa0d409ff/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.613279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b58d9-db49-412f-b4f7-09d9c27c60a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.614017 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b58d9-db49-412f-b4f7-09d9c27c60a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.614308 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b58d9-db49-412f-b4f7-09d9c27c60a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.614569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b58d9-db49-412f-b4f7-09d9c27c60a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.615264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b58d9-db49-412f-b4f7-09d9c27c60a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.617241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b58d9-db49-412f-b4f7-09d9c27c60a7 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.641454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49d79a-37b1-49a4-a662-c14afdb02530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad847618ee09484a96cda3bd83da358f, user_id=0418a67fb0114b809b01ee6fa0d409ff, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.648290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49d79a-37b1-49a4-a662-c14afdb02530 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.674770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 808/1622] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:31:44 2022] POST /identity/v3/users => generated 288 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:44.675964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b58d9-db49-412f-b4f7-09d9c27c60a7 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.676998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 815/1623] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:44 2022] PUT /identity/v3/projects/ad847618ee09484a96cda3bd83da358f/users/0418a67fb0114b809b01ee6fa0d409ff/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:44.680901 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49d79a-37b1-49a4-a662-c14afdb02530 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.695200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b58d9-db49-412f-b4f7-09d9c27c60a7 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.726820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:44.764293 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49d79a-37b1-49a4-a662-c14afdb02530 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.770037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141fb55-9080-4323-8135-2b127bfbb752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.770713 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141fb55-9080-4323-8135-2b127bfbb752 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.770994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141fb55-9080-4323-8135-2b127bfbb752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.771285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141fb55-9080-4323-8135-2b127bfbb752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.772247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a141fb55-9080-4323-8135-2b127bfbb752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.774325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a141fb55-9080-4323-8135-2b127bfbb752 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.789894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 816/1624] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:44.790714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141fb55-9080-4323-8135-2b127bfbb752 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:44.802145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141fb55-9080-4323-8135-2b127bfbb752 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:44.808242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920b8a7-13f2-475b-a76b-16f0d29b1be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.810328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920b8a7-13f2-475b-a76b-16f0d29b1be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.810746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920b8a7-13f2-475b-a76b-16f0d29b1be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.811201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920b8a7-13f2-475b-a76b-16f0d29b1be7 admin admin] PATH_INFO: `/v3/projects/ad847618ee09484a96cda3bd83da358f/users/0418a67fb0114b809b01ee6fa0d409ff/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.813954 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920b8a7-13f2-475b-a76b-16f0d29b1be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad847618ee09484a96cda3bd83da358f, user_id=0418a67fb0114b809b01ee6fa0d409ff, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.817681 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920b8a7-13f2-475b-a76b-16f0d29b1be7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.847248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 817/1625] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:44 2022] PUT /identity/v3/projects/ad847618ee09484a96cda3bd83da358f/users/0418a67fb0114b809b01ee6fa0d409ff/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:44.849520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cd933-58ca-400d-a213-2bc93e8d389e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.849827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cd933-58ca-400d-a213-2bc93e8d389e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.850601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cd933-58ca-400d-a213-2bc93e8d389e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.850716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3c3b9-c330-4637-a47a-3ea15d41c9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:44.851340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3c3b9-c330-4637-a47a-3ea15d41c9cf admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:44.851633 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3c3b9-c330-4637-a47a-3ea15d41c9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:44.851931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3c3b9-c330-4637-a47a-3ea15d41c9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:44.852575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3c3b9-c330-4637-a47a-3ea15d41c9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:44.856739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3c3b9-c330-4637-a47a-3ea15d41c9cf admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:44.876273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 809/1626] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:44.877254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3c3b9-c330-4637-a47a-3ea15d41c9cf admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:45.002971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f77cd933-58ca-400d-a213-2bc93e8d389e None None] MFA Rules not processed for user `0418a67fb0114b809b01ee6fa0d409ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:45.002971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f77cd933-58ca-400d-a213-2bc93e8d389e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:45.008022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3c3b9-c330-4637-a47a-3ea15d41c9cf admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:45.060472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:45.113705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d36c3-f80f-4fa8-89c5-58d5e265fa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:45.119331 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d36c3-f80f-4fa8-89c5-58d5e265fa7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:45.120900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d36c3-f80f-4fa8-89c5-58d5e265fa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:45.121368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d36c3-f80f-4fa8-89c5-58d5e265fa7f admin admin] PATH_INFO: `/v3/projects/920e6698819f424b96b2ff3bb357d031/users/5d6847c566af44519d7710a9f213268c/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:45.125518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 818/1627] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:45.139028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cd933-58ca-400d-a213-2bc93e8d389e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:45.176188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d36c3-f80f-4fa8-89c5-58d5e265fa7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=920e6698819f424b96b2ff3bb357d031, user_id=5d6847c566af44519d7710a9f213268c, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:45.183475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d36c3-f80f-4fa8-89c5-58d5e265fa7f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:45.213627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 810/1628] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:44 2022] PUT /identity/v3/projects/920e6698819f424b96b2ff3bb357d031/users/5d6847c566af44519d7710a9f213268c/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:45.219191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d36c3-f80f-4fa8-89c5-58d5e265fa7f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:45.266521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cd933-58ca-400d-a213-2bc93e8d389e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:45.328950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d36c3-f80f-4fa8-89c5-58d5e265fa7f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:45.336125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e566c-bd2e-4aed-88a7-08f10547a6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:45.337130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e566c-bd2e-4aed-88a7-08f10547a6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:45.337495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e566c-bd2e-4aed-88a7-08f10547a6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:45.338347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e566c-bd2e-4aed-88a7-08f10547a6e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:45.339143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e566c-bd2e-4aed-88a7-08f10547a6e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:45.341834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e566c-bd2e-4aed-88a7-08f10547a6e2 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:45.358313 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 811/1629] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:31:45.362506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e566c-bd2e-4aed-88a7-08f10547a6e2 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:45.372936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f45955-6a11-40eb-bc49-a4baaf6de66e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:45.374152 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f45955-6a11-40eb-bc49-a4baaf6de66e service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:45.375275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f45955-6a11-40eb-bc49-a4baaf6de66e service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:45.378133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f45955-6a11-40eb-bc49-a4baaf6de66e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:45.382632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e566c-bd2e-4aed-88a7-08f10547a6e2 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:45.397682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab18893-446e-4737-8abc-bcefaee1d13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:45.398373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab18893-446e-4737-8abc-bcefaee1d13c admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:45.400797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab18893-446e-4737-8abc-bcefaee1d13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:45.401115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab18893-446e-4737-8abc-bcefaee1d13c admin admin] PATH_INFO: `/v3/projects/920e6698819f424b96b2ff3bb357d031/users/5d6847c566af44519d7710a9f213268c/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:45.405707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab18893-446e-4737-8abc-bcefaee1d13c admin admin] RBAC: Authorizing `identity:create_grant(project_id=920e6698819f424b96b2ff3bb357d031, user_id=5d6847c566af44519d7710a9f213268c, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:45.412925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab18893-446e-4737-8abc-bcefaee1d13c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:45.443790 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 812/1630] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:45 2022] PUT /identity/v3/projects/920e6698819f424b96b2ff3bb357d031/users/5d6847c566af44519d7710a9f213268c/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:45.455760 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ad7e6-8f0c-4296-8e7a-0b5c3c143e9f None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:45.456049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ad7e6-8f0c-4296-8e7a-0b5c3c143e9f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:45.456864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ad7e6-8f0c-4296-8e7a-0b5c3c143e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:45.519282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3b9ad7e6-8f0c-4296-8e7a-0b5c3c143e9f None None] MFA Rules not processed for user `5d6847c566af44519d7710a9f213268c`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:45.522052 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3b9ad7e6-8f0c-4296-8e7a-0b5c3c143e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:45.529457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f45955-6a11-40eb-bc49-a4baaf6de66e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:45.531093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f45955-6a11-40eb-bc49-a4baaf6de66e service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:45.531900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:45.551302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 819/1631] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:45.552870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c1c80-6488-435d-91b7-aa97308c6e62 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:45.553323 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c1c80-6488-435d-91b7-aa97308c6e62 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:45.553709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c1c80-6488-435d-91b7-aa97308c6e62 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:45.562826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 820/1632] 10.222.0.55 () {54 vars in 962 bytes} [Fri Aug 19 16:31:45 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:31:45.566102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41f095-0888-4285-8d54-7ed23bbc996f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:45.566598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41f095-0888-4285-8d54-7ed23bbc996f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:45.566891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41f095-0888-4285-8d54-7ed23bbc996f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:45.676123 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-aa41f095-0888-4285-8d54-7ed23bbc996f None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:45.677404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-aa41f095-0888-4285-8d54-7ed23bbc996f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:45.705574 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 813/1633] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:45.722123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ad7e6-8f0c-4296-8e7a-0b5c3c143e9f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:45.773811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:45.848472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ad7e6-8f0c-4296-8e7a-0b5c3c143e9f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:45.859680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f6349-d521-4584-927f-5c35dd4d635c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:45.860394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f6349-d521-4584-927f-5c35dd4d635c service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:45.860671 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f6349-d521-4584-927f-5c35dd4d635c service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:45.861102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f6349-d521-4584-927f-5c35dd4d635c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:45.882012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f6349-d521-4584-927f-5c35dd4d635c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:45.882290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 821/1634] 10.222.0.55 () {58 vars in 1059 bytes} [Fri Aug 19 16:31:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:45.883302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41f095-0888-4285-8d54-7ed23bbc996f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:45.883690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f6349-d521-4584-927f-5c35dd4d635c service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:45.901703 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41f095-0888-4285-8d54-7ed23bbc996f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:45.905929 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 814/1635] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:31:45.907236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4975ef-3050-4622-85e6-3c4eecbcf377 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:45.907644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4975ef-3050-4622-85e6-3c4eecbcf377 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:45.907924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4975ef-3050-4622-85e6-3c4eecbcf377 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:45.925167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 815/1636] 10.222.0.55 () {54 vars in 912 bytes} [Fri Aug 19 16:31:45 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:31:45.926357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5c43c-f7ff-4232-a522-ecd47824456b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:45.928535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d31ea3-98b0-404f-a406-eee7a773c20a None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:45.929117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5c43c-f7ff-4232-a522-ecd47824456b service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:45.929400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5c43c-f7ff-4232-a522-ecd47824456b service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:45.929704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d31ea3-98b0-404f-a406-eee7a773c20a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:45.930918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5c43c-f7ff-4232-a522-ecd47824456b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:45.931381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d31ea3-98b0-404f-a406-eee7a773c20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:45.979332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-92d31ea3-98b0-404f-a406-eee7a773c20a None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:45.982882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-92d31ea3-98b0-404f-a406-eee7a773c20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:46.012951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 816/1637] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:31:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:46.083909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d31ea3-98b0-404f-a406-eee7a773c20a None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:46.093959 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe5c43c-f7ff-4232-a522-ecd47824456b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:46.096122 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe5c43c-f7ff-4232-a522-ecd47824456b service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:46.099719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d31ea3-98b0-404f-a406-eee7a773c20a None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:46.111779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fcbda-d21b-4436-8061-27008ccb5607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:46.113226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fcbda-d21b-4436-8061-27008ccb5607 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:46.114089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 822/1638] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:46.114579 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fcbda-d21b-4436-8061-27008ccb5607 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:46.115561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fcbda-d21b-4436-8061-27008ccb5607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:46.132794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fcbda-d21b-4436-8061-27008ccb5607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:46.134889 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fcbda-d21b-4436-8061-27008ccb5607 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:46.155655 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 817/1639] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:31:47.209546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5c43c-f7ff-4232-a522-ecd47824456b service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:47.321554 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5c43c-f7ff-4232-a522-ecd47824456b service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:47.331542 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:47.333083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:47.333668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:47.334129 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] PATH_INFO: `/v3/users/072cfcc77e9f41e0b0bbe67729b74be8` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:47.337245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=072cfcc77e9f41e0b0bbe67729b74be8)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:47.338832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:31:47.338832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Aug 19 16:31:47.338832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: Aug 19 16:31:47.341750 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:47.458125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '072cfcc77e9f41e0b0bbe67729b74be8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:47.459029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '072cfcc77e9f41e0b0bbe67729b74be8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:47.459919 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '072cfcc77e9f41e0b0bbe67729b74be8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:47.498205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '072cfcc77e9f41e0b0bbe67729b74be8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:47.504695 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:47.530567 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '072cfcc77e9f41e0b0bbe67729b74be8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:47.555334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 823/1640] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:31:47 2022] DELETE /identity/v3/users/072cfcc77e9f41e0b0bbe67729b74be8 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:47.556592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e719549-d4b2-46a9-9983-46f73307666a None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:47.557133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e719549-d4b2-46a9-9983-46f73307666a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:47.557216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e719549-d4b2-46a9-9983-46f73307666a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:47.589689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-7e719549-d4b2-46a9-9983-46f73307666a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:47.601532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7e719549-d4b2-46a9-9983-46f73307666a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:47.753249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 818/1641] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:47.798513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:47.800923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:47.966052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f918dcf-4665-448b-b9e1-b3e402e6db63 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:47.970257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7042b6-ba76-42b7-bb34-5f5899a2882b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:47.971732 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7042b6-ba76-42b7-bb34-5f5899a2882b service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:47.972198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7042b6-ba76-42b7-bb34-5f5899a2882b service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:47.972644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7042b6-ba76-42b7-bb34-5f5899a2882b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:47.983653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7042b6-ba76-42b7-bb34-5f5899a2882b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:47.985673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7042b6-ba76-42b7-bb34-5f5899a2882b service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:48.002973 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 824/1642] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:48.676923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75601166-30db-472a-92ca-dbdb01c5166d None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:48.676923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75601166-30db-472a-92ca-dbdb01c5166d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:48.676923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75601166-30db-472a-92ca-dbdb01c5166d None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:48.676923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 819/1643] 10.222.0.55 () {54 vars in 962 bytes} [Fri Aug 19 16:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:31:48.687547 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da933f-8513-465c-a20d-44e07dd63b05 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:48.687547 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da933f-8513-465c-a20d-44e07dd63b05 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:48.687547 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da933f-8513-465c-a20d-44e07dd63b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:48.728650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-43da933f-8513-465c-a20d-44e07dd63b05 None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:48.731914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-43da933f-8513-465c-a20d-44e07dd63b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:48.850894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 825/1644] 10.222.0.55 () {58 vars in 1059 bytes} [Fri Aug 19 16:31:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:48.856922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbfe9a-4366-42ce-9b8f-f8af375d40c6 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:48.857516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbfe9a-4366-42ce-9b8f-f8af375d40c6 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:48.857865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbfe9a-4366-42ce-9b8f-f8af375d40c6 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:48.859893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 820/1645] 10.222.0.55 () {54 vars in 912 bytes} [Fri Aug 19 16:31:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:31:48.866527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b1812-e05f-4401-b81c-df507ca3f675 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:48.867164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b1812-e05f-4401-b81c-df507ca3f675 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:48.867838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b1812-e05f-4401-b81c-df507ca3f675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:48.951672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c81b1812-e05f-4401-b81c-df507ca3f675 None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:48.953443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c81b1812-e05f-4401-b81c-df507ca3f675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:48.972465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 826/1646] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:31:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:48.992029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbfe9a-4366-42ce-9b8f-f8af375d40c6 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:48.996405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-fffbfe9a-4366-42ce-9b8f-f8af375d40c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:49.057590 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:49.139686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbfe9a-4366-42ce-9b8f-f8af375d40c6 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:49.147167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b003c5f-fb5a-4c30-940c-1631223826f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:49.147978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b003c5f-fb5a-4c30-940c-1631223826f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:49.148905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b003c5f-fb5a-4c30-940c-1631223826f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:49.149288 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b003c5f-fb5a-4c30-940c-1631223826f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:49.167614 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b003c5f-fb5a-4c30-940c-1631223826f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:49.172971 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b003c5f-fb5a-4c30-940c-1631223826f4 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:49.217151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 821/1647] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:50.266104 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b1812-e05f-4401-b81c-df507ca3f675 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:50.267768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c81b1812-e05f-4401-b81c-df507ca3f675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:50.298878 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b1812-e05f-4401-b81c-df507ca3f675 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:50.304303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:50.305062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:50.305339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:50.305611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] PATH_INFO: `/v3/projects/d92de66a9f7543eb8c917c1a6ddfcd55` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:50.306763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92de66a9f7543eb8c917c1a6ddfcd55)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:50.309395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:50.395796 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92de66a9f7543eb8c917c1a6ddfcd55', 'request_id': 'req-1e72ba6a-488e-4e78-96a5-6a851002cb3c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:50.405097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92de66a9f7543eb8c917c1a6ddfcd55', 'request_id': 'req-1e72ba6a-488e-4e78-96a5-6a851002cb3c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:50.417272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92de66a9f7543eb8c917c1a6ddfcd55', 'request_id': 'req-1e72ba6a-488e-4e78-96a5-6a851002cb3c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:50.422754 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:50.448029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] The token cache is being invalidate because project d92de66a9f7543eb8c917c1a6ddfcd55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:31:50.450747 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:50.452410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:50.454549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 827/1648] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:31:50 2022] DELETE /identity/v3/projects/d92de66a9f7543eb8c917c1a6ddfcd55 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:50.458549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b003c5f-fb5a-4c30-940c-1631223826f4 service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:50.476079 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b003c5f-fb5a-4c30-940c-1631223826f4 service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:50.707839 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae2755-94f6-41e2-88e5-d8cb5ca1bbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:50.710359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae2755-94f6-41e2-88e5-d8cb5ca1bbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:50.710646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae2755-94f6-41e2-88e5-d8cb5ca1bbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:50.710924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae2755-94f6-41e2-88e5-d8cb5ca1bbf5 admin admin] PATH_INFO: `/v3/users/5d6847c566af44519d7710a9f213268c` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:50.714335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae2755-94f6-41e2-88e5-d8cb5ca1bbf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d6847c566af44519d7710a9f213268c)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:50.720474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae2755-94f6-41e2-88e5-d8cb5ca1bbf5 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:50.880269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-53ae2755-94f6-41e2-88e5-d8cb5ca1bbf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d6847c566af44519d7710a9f213268c'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:50.892741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-53ae2755-94f6-41e2-88e5-d8cb5ca1bbf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d6847c566af44519d7710a9f213268c'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:50.896078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-53ae2755-94f6-41e2-88e5-d8cb5ca1bbf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d6847c566af44519d7710a9f213268c'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:50.916000 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-53ae2755-94f6-41e2-88e5-d8cb5ca1bbf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d6847c566af44519d7710a9f213268c'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:50.960932 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-53ae2755-94f6-41e2-88e5-d8cb5ca1bbf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d6847c566af44519d7710a9f213268c'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:51.019656 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 822/1649] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:31:50 2022] DELETE /identity/v3/users/5d6847c566af44519d7710a9f213268c => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:54.113111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:54.114738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:54.191015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72ba6a-488e-4e78-96a5-6a851002cb3c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:54.286341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f4651-429e-4bbc-ba46-992ad4a85c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:54.287016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f4651-429e-4bbc-ba46-992ad4a85c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:54.287297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f4651-429e-4bbc-ba46-992ad4a85c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:54.287575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f4651-429e-4bbc-ba46-992ad4a85c21 admin admin] PATH_INFO: `/v3/projects/920e6698819f424b96b2ff3bb357d031` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:54.288806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f4651-429e-4bbc-ba46-992ad4a85c21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=920e6698819f424b96b2ff3bb357d031)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:54.290785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f4651-429e-4bbc-ba46-992ad4a85c21 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:54.357087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-053f4651-429e-4bbc-ba46-992ad4a85c21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '920e6698819f424b96b2ff3bb357d031', 'request_id': 'req-053f4651-429e-4bbc-ba46-992ad4a85c21'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:54.363530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-053f4651-429e-4bbc-ba46-992ad4a85c21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '920e6698819f424b96b2ff3bb357d031', 'request_id': 'req-053f4651-429e-4bbc-ba46-992ad4a85c21'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:54.372847 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-053f4651-429e-4bbc-ba46-992ad4a85c21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '920e6698819f424b96b2ff3bb357d031', 'request_id': 'req-053f4651-429e-4bbc-ba46-992ad4a85c21'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:54.408282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-053f4651-429e-4bbc-ba46-992ad4a85c21 admin admin] The token cache is being invalidate because project 920e6698819f424b96b2ff3bb357d031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:31:54.409014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-053f4651-429e-4bbc-ba46-992ad4a85c21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:54.410491 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-053f4651-429e-4bbc-ba46-992ad4a85c21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:54.421874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3a1b7-4ed2-4a69-8905-5be19d11401b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:54.422700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3a1b7-4ed2-4a69-8905-5be19d11401b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:54.422700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3a1b7-4ed2-4a69-8905-5be19d11401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:54.422856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 828/1650] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:31:54 2022] DELETE /identity/v3/projects/920e6698819f424b96b2ff3bb357d031 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:54.484549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-8ae3a1b7-4ed2-4a69-8905-5be19d11401b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:54.487551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8ae3a1b7-4ed2-4a69-8905-5be19d11401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:54.558111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:54.580537 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 823/1651] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:54.589697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2129599-5af3-47fa-af9f-1dd2239633c2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:54.590298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2129599-5af3-47fa-af9f-1dd2239633c2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:54.590651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2129599-5af3-47fa-af9f-1dd2239633c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:54.637231 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c2129599-5af3-47fa-af9f-1dd2239633c2 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:54.638790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c2129599-5af3-47fa-af9f-1dd2239633c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:54.649384 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 829/1652] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:54.654147 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3a1b7-4ed2-4a69-8905-5be19d11401b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:54.666494 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3a1b7-4ed2-4a69-8905-5be19d11401b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:54.674295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbf92e-d239-417a-bc1c-977f3e0395ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:54.675212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbf92e-d239-417a-bc1c-977f3e0395ed admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:54.675855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbf92e-d239-417a-bc1c-977f3e0395ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:54.676322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbf92e-d239-417a-bc1c-977f3e0395ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:54.677400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbf92e-d239-417a-bc1c-977f3e0395ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:31:54.677871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbf92e-d239-417a-bc1c-977f3e0395ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:54.679663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbf92e-d239-417a-bc1c-977f3e0395ed admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:54.693507 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 824/1653] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:31:54.699793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2129599-5af3-47fa-af9f-1dd2239633c2 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:54.716085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2129599-5af3-47fa-af9f-1dd2239633c2 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:54.738161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17701eaa-c9a1-4a49-8c4e-2712e30b28c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:54.740006 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17701eaa-c9a1-4a49-8c4e-2712e30b28c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:54.740006 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17701eaa-c9a1-4a49-8c4e-2712e30b28c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:54.740184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17701eaa-c9a1-4a49-8c4e-2712e30b28c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:54.740708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17701eaa-c9a1-4a49-8c4e-2712e30b28c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:54.742713 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17701eaa-c9a1-4a49-8c4e-2712e30b28c5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:54.803266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 830/1654] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:31:54 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:54.807581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbf92e-d239-417a-bc1c-977f3e0395ed admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:54.823871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbf92e-d239-417a-bc1c-977f3e0395ed admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:54.956407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157a4e7-1640-4b24-ad40-3de46e46718a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:54.957076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157a4e7-1640-4b24-ad40-3de46e46718a admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:54.957360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157a4e7-1640-4b24-ad40-3de46e46718a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:54.957693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157a4e7-1640-4b24-ad40-3de46e46718a admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:54.958394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157a4e7-1640-4b24-ad40-3de46e46718a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:54.961452 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157a4e7-1640-4b24-ad40-3de46e46718a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:55.055154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 825/1655] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:31:54 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:55.056676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17701eaa-c9a1-4a49-8c4e-2712e30b28c5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:55.077917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17701eaa-c9a1-4a49-8c4e-2712e30b28c5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:55.086674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08acc940-b1f2-42b9-b734-9b1c8b7bcd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:55.086913 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08acc940-b1f2-42b9-b734-9b1c8b7bcd0f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:55.087883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08acc940-b1f2-42b9-b734-9b1c8b7bcd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:55.088249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08acc940-b1f2-42b9-b734-9b1c8b7bcd0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:55.089042 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08acc940-b1f2-42b9-b734-9b1c8b7bcd0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:55.093474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08acc940-b1f2-42b9-b734-9b1c8b7bcd0f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:55.133710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 831/1656] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:55.138405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157a4e7-1640-4b24-ad40-3de46e46718a admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:55.164930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157a4e7-1640-4b24-ad40-3de46e46718a admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:55.175916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b15bb8-b3ef-43a4-b7d0-fcb374471fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:55.176554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b15bb8-b3ef-43a4-b7d0-fcb374471fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:55.176863 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b15bb8-b3ef-43a4-b7d0-fcb374471fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:55.177186 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b15bb8-b3ef-43a4-b7d0-fcb374471fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:55.178035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b15bb8-b3ef-43a4-b7d0-fcb374471fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:55.195493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b15bb8-b3ef-43a4-b7d0-fcb374471fe7 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:55.226962 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 826/1657] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:55.229354 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08acc940-b1f2-42b9-b734-9b1c8b7bcd0f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:55.259964 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08acc940-b1f2-42b9-b734-9b1c8b7bcd0f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:55.286953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21af80c-8681-4e5a-8b2f-c92aeddc4380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:55.287651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21af80c-8681-4e5a-8b2f-c92aeddc4380 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:55.288420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21af80c-8681-4e5a-8b2f-c92aeddc4380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:55.288420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21af80c-8681-4e5a-8b2f-c92aeddc4380 admin admin] PATH_INFO: `/v3/projects/e9128b595c6d482fa86897ad44d80fff/users/d3ed0ed22104437d996653edc67f5fdf/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:55.340526 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21af80c-8681-4e5a-8b2f-c92aeddc4380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9128b595c6d482fa86897ad44d80fff, user_id=d3ed0ed22104437d996653edc67f5fdf, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:55.347529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21af80c-8681-4e5a-8b2f-c92aeddc4380 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:55.408105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 832/1658] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:55 2022] PUT /identity/v3/projects/e9128b595c6d482fa86897ad44d80fff/users/d3ed0ed22104437d996653edc67f5fdf/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:55.412057 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390f493-10b6-4d8b-8621-bf74161c3a98 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:55.412350 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390f493-10b6-4d8b-8621-bf74161c3a98 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:55.412652 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390f493-10b6-4d8b-8621-bf74161c3a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:55.483463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-9390f493-10b6-4d8b-8621-bf74161c3a98 None None] MFA Rules not processed for user `d3ed0ed22104437d996653edc67f5fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:55.484693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9390f493-10b6-4d8b-8621-bf74161c3a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:55.720657 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 827/1659] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:55.726664 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21af80c-8681-4e5a-8b2f-c92aeddc4380 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:55.794082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:55.914017 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21af80c-8681-4e5a-8b2f-c92aeddc4380 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:55.937073 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da821300-5f71-495a-97d9-38343dd64a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:55.937892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da821300-5f71-495a-97d9-38343dd64a6f admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:55.939309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da821300-5f71-495a-97d9-38343dd64a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:55.939767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da821300-5f71-495a-97d9-38343dd64a6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:55.942635 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da821300-5f71-495a-97d9-38343dd64a6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:55.950987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da821300-5f71-495a-97d9-38343dd64a6f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:56.039377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 833/1660] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:31:55 2022] POST /identity/v3/projects => generated 381 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:56.043382 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390f493-10b6-4d8b-8621-bf74161c3a98 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:56.063649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390f493-10b6-4d8b-8621-bf74161c3a98 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:56.177277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:56.218105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e47906-b1aa-4aad-b675-9ce571ba0f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:56.218746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e47906-b1aa-4aad-b675-9ce571ba0f20 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:56.219025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e47906-b1aa-4aad-b675-9ce571ba0f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:56.219299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e47906-b1aa-4aad-b675-9ce571ba0f20 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:56.222023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e47906-b1aa-4aad-b675-9ce571ba0f20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:56.226615 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e47906-b1aa-4aad-b675-9ce571ba0f20 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:56.285960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 828/1661] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:31:56 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:31:56.289678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da821300-5f71-495a-97d9-38343dd64a6f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:56.303462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da821300-5f71-495a-97d9-38343dd64a6f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:56.318124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecb668-7703-401c-a8b6-f120c7af1ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:56.318794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecb668-7703-401c-a8b6-f120c7af1ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:56.321260 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecb668-7703-401c-a8b6-f120c7af1ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:56.322722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecb668-7703-401c-a8b6-f120c7af1ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:56.322722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecb668-7703-401c-a8b6-f120c7af1ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:56.326944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecb668-7703-401c-a8b6-f120c7af1ca5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:56.350906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 834/1662] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:31:56.357161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e47906-b1aa-4aad-b675-9ce571ba0f20 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:56.375980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e47906-b1aa-4aad-b675-9ce571ba0f20 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:56.381477 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c76d92-1c5d-4341-945c-13c4a459abf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:56.382465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c76d92-1c5d-4341-945c-13c4a459abf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:56.382761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c76d92-1c5d-4341-945c-13c4a459abf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:56.383036 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c76d92-1c5d-4341-945c-13c4a459abf2 admin admin] PATH_INFO: `/v3/projects/6e5563fb60c2429d86350b30d5f7ff7b/users/462d0ac51bd644d3860a2310f524d034/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:56.415877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c76d92-1c5d-4341-945c-13c4a459abf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5563fb60c2429d86350b30d5f7ff7b, user_id=462d0ac51bd644d3860a2310f524d034, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:56.419214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c76d92-1c5d-4341-945c-13c4a459abf2 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:56.464304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 829/1663] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:56 2022] PUT /identity/v3/projects/6e5563fb60c2429d86350b30d5f7ff7b/users/462d0ac51bd644d3860a2310f524d034/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:56.466853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecb668-7703-401c-a8b6-f120c7af1ca5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:56.479426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecb668-7703-401c-a8b6-f120c7af1ca5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:56.605570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deee7f3-0c48-4595-97fc-24d3da501cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:56.606419 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deee7f3-0c48-4595-97fc-24d3da501cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:56.606698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deee7f3-0c48-4595-97fc-24d3da501cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:56.606971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deee7f3-0c48-4595-97fc-24d3da501cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:56.607662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deee7f3-0c48-4595-97fc-24d3da501cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:56.610298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deee7f3-0c48-4595-97fc-24d3da501cd8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:56.641289 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08c393-3b2f-4a34-bfeb-1cf8eb97be30 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:56.642125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08c393-3b2f-4a34-bfeb-1cf8eb97be30 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:56.642541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08c393-3b2f-4a34-bfeb-1cf8eb97be30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:56.647899 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 835/1664] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:31:56.649261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deee7f3-0c48-4595-97fc-24d3da501cd8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:56.680282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deee7f3-0c48-4595-97fc-24d3da501cd8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:56.686588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980e236-2dbd-4b0d-87cf-39f7dc387e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:56.687453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980e236-2dbd-4b0d-87cf-39f7dc387e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:56.687667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980e236-2dbd-4b0d-87cf-39f7dc387e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:56.688009 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980e236-2dbd-4b0d-87cf-39f7dc387e7e admin admin] PATH_INFO: `/v3/projects/6e5563fb60c2429d86350b30d5f7ff7b/users/462d0ac51bd644d3860a2310f524d034/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:56.690336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980e236-2dbd-4b0d-87cf-39f7dc387e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5563fb60c2429d86350b30d5f7ff7b, user_id=462d0ac51bd644d3860a2310f524d034, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:56.694894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980e236-2dbd-4b0d-87cf-39f7dc387e7e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:56.712684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 836/1665] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:31:56 2022] PUT /identity/v3/projects/6e5563fb60c2429d86350b30d5f7ff7b/users/462d0ac51bd644d3860a2310f524d034/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:56.722148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-eb08c393-3b2f-4a34-bfeb-1cf8eb97be30 None None] MFA Rules not processed for user `e09fd34737af41dc972d806dc3b0705e`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:56.723782 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-eb08c393-3b2f-4a34-bfeb-1cf8eb97be30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:56.736657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d839eb-7548-4feb-90db-79d5d1906793 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:56.738904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d839eb-7548-4feb-90db-79d5d1906793 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:56.739426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d839eb-7548-4feb-90db-79d5d1906793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:56.812694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-06d839eb-7548-4feb-90db-79d5d1906793 None None] MFA Rules not processed for user `462d0ac51bd644d3860a2310f524d034`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:56.814125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-06d839eb-7548-4feb-90db-79d5d1906793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:56.848568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 830/1666] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:56.867935 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08c393-3b2f-4a34-bfeb-1cf8eb97be30 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:56.869940 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-eb08c393-3b2f-4a34-bfeb-1cf8eb97be30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:57.023583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 837/1667] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:57.035930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d839eb-7548-4feb-90db-79d5d1906793 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:57.038929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-06d839eb-7548-4feb-90db-79d5d1906793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:57.054527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:57.098379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08c393-3b2f-4a34-bfeb-1cf8eb97be30 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:57.114793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f4b3a-6ab5-47a5-bcae-baf93680c3d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:57.115655 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f4b3a-6ab5-47a5-bcae-baf93680c3d1 service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:57.116090 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f4b3a-6ab5-47a5-bcae-baf93680c3d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:57.116536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f4b3a-6ab5-47a5-bcae-baf93680c3d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:57.144577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f4b3a-6ab5-47a5-bcae-baf93680c3d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:57.146481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f4b3a-6ab5-47a5-bcae-baf93680c3d1 service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:57.166308 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 831/1668] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:57.180935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d839eb-7548-4feb-90db-79d5d1906793 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:57.185899 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393fe7a-44f7-4f7f-a92c-1013069eafe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:57.186938 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393fe7a-44f7-4f7f-a92c-1013069eafe7 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:57.187420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393fe7a-44f7-4f7f-a92c-1013069eafe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:57.187900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393fe7a-44f7-4f7f-a92c-1013069eafe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:57.209831 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393fe7a-44f7-4f7f-a92c-1013069eafe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:57.211935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393fe7a-44f7-4f7f-a92c-1013069eafe7 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:57.242879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 838/1669] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:57.367994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f4b3a-6ab5-47a5-bcae-baf93680c3d1 service nova] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:57.370293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-049f4b3a-6ab5-47a5-bcae-baf93680c3d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:57.435674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:57.441289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393fe7a-44f7-4f7f-a92c-1013069eafe7 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:57.464177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393fe7a-44f7-4f7f-a92c-1013069eafe7 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:57.477889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76273276-542e-49a9-81f0-8d83e32b5d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:57.479694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76273276-542e-49a9-81f0-8d83e32b5d18 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:57.480676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76273276-542e-49a9-81f0-8d83e32b5d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:57.481654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76273276-542e-49a9-81f0-8d83e32b5d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:57.509589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f4b3a-6ab5-47a5-bcae-baf93680c3d1 service nova] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:57.520889 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:57.521547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:57.521826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:57.522097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] PATH_INFO: `/v3/users/e2ada95813374bdf894a1ffcbbe1c7da` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:57.525946 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ada95813374bdf894a1ffcbbe1c7da)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:57.528170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:57.643402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ada95813374bdf894a1ffcbbe1c7da'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:57.645732 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ada95813374bdf894a1ffcbbe1c7da'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:57.648392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ada95813374bdf894a1ffcbbe1c7da'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:57.656071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76273276-542e-49a9-81f0-8d83e32b5d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:57.659920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76273276-542e-49a9-81f0-8d83e32b5d18 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:57.667386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ada95813374bdf894a1ffcbbe1c7da'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:57.704298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 839/1670] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:57.709458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ada95813374bdf894a1ffcbbe1c7da'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:57.734442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 832/1671] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:31:57 2022] DELETE /identity/v3/users/e2ada95813374bdf894a1ffcbbe1c7da => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:57.737304 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2ece7-c79c-4ec1-a88f-17ed6d5545b8 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:57.739339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2ece7-c79c-4ec1-a88f-17ed6d5545b8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:57.739339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2ece7-c79c-4ec1-a88f-17ed6d5545b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:57.800548 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6da2ece7-c79c-4ec1-a88f-17ed6d5545b8 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:57.803510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6da2ece7-c79c-4ec1-a88f-17ed6d5545b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:58.034271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 840/1672] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:58.056115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:58.060254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:58.155219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93697c96-c8ef-4d38-93f4-76337bd5da0a admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:58.160554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcf97c-544d-498b-afd2-0cbe05ba8fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:58.162080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcf97c-544d-498b-afd2-0cbe05ba8fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:58.162576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcf97c-544d-498b-afd2-0cbe05ba8fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:58.163116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcf97c-544d-498b-afd2-0cbe05ba8fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:58.191877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebcf97c-544d-498b-afd2-0cbe05ba8fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:58.193905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebcf97c-544d-498b-afd2-0cbe05ba8fb6 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:58.210426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 833/1673] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:58.270880 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2ece7-c79c-4ec1-a88f-17ed6d5545b8 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:58.272613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6da2ece7-c79c-4ec1-a88f-17ed6d5545b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:58.291828 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2ece7-c79c-4ec1-a88f-17ed6d5545b8 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:58.300620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:58.301433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:58.301968 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:58.302262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] PATH_INFO: `/v3/users/d3ed0ed22104437d996653edc67f5fdf` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:58.303684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ed0ed22104437d996653edc67f5fdf)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:58.305911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:58.400325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:58.423810 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ed0ed22104437d996653edc67f5fdf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:58.425244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ed0ed22104437d996653edc67f5fdf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:58.426698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ed0ed22104437d996653edc67f5fdf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:58.448027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ed0ed22104437d996653edc67f5fdf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:58.486652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ed0ed22104437d996653edc67f5fdf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:58.546470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ce2c9-afad-4926-965c-647ea81c634c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:58.546903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ce2c9-afad-4926-965c-647ea81c634c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:58.547355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ce2c9-afad-4926-965c-647ea81c634c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:58.551029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 841/1674] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:31:58 2022] DELETE /identity/v3/users/d3ed0ed22104437d996653edc67f5fdf => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:58.626467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-7e0ce2c9-afad-4926-965c-647ea81c634c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:58.632053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7e0ce2c9-afad-4926-965c-647ea81c634c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:58.768198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:31:58.825095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 834/1675] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:31:58.895089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:58.897039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:59.045655 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d94df-66f6-4968-bc63-fa829cdf8e19 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:59.070794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3c76f-de54-4fde-8c6e-45d8c0e2e8b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:59.072821 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3c76f-de54-4fde-8c6e-45d8c0e2e8b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:59.073532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3c76f-de54-4fde-8c6e-45d8c0e2e8b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:59.074784 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3c76f-de54-4fde-8c6e-45d8c0e2e8b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:59.095577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3c76f-de54-4fde-8c6e-45d8c0e2e8b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:59.097124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3c76f-de54-4fde-8c6e-45d8c0e2e8b0 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:59.114559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 842/1676] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:31:59.491251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ce2c9-afad-4926-965c-647ea81c634c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:31:59.492986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7e0ce2c9-afad-4926-965c-647ea81c634c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:31:59.514508 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ce2c9-afad-4926-965c-647ea81c634c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:31:59.528732 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:31:59.530697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:59.531085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:59.533272 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] PATH_INFO: `/v3/projects/e50130a56a8a4cd5be25f7350d22cf05` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:59.535221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e50130a56a8a4cd5be25f7350d22cf05)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:31:59.537094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: WARNING py.warnings [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 16:31:59.537094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: warnings.warn(msg) Aug 19 16:31:59.537094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: Aug 19 16:31:59.552087 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:31:59.727968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e50130a56a8a4cd5be25f7350d22cf05', 'request_id': 'req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:59.792601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e50130a56a8a4cd5be25f7350d22cf05', 'request_id': 'req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:59.825899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e50130a56a8a4cd5be25f7350d22cf05', 'request_id': 'req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:59.853625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] The token cache is being invalidate because project e50130a56a8a4cd5be25f7350d22cf05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:31:59.854349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:59.856818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-5f2ea4d8-a4b5-4371-84e5-7374166cdeb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:31:59.859489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 835/1677] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:31:59 2022] DELETE /identity/v3/projects/e50130a56a8a4cd5be25f7350d22cf05 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:31:59.866930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58502f32-f822-47ec-983a-37a40ff3f2a8 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:31:59.869665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58502f32-f822-47ec-983a-37a40ff3f2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:31:59.870347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58502f32-f822-47ec-983a-37a40ff3f2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:31:59.937625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-58502f32-f822-47ec-983a-37a40ff3f2a8 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:31:59.939598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-58502f32-f822-47ec-983a-37a40ff3f2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:00.131197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 843/1678] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:00.139404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5865cf5-3e31-4a06-bf6a-2ccf83a4e71b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:00.140118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5865cf5-3e31-4a06-bf6a-2ccf83a4e71b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:00.140515 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5865cf5-3e31-4a06-bf6a-2ccf83a4e71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:00.177186 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f5865cf5-3e31-4a06-bf6a-2ccf83a4e71b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:00.179132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f5865cf5-3e31-4a06-bf6a-2ccf83a4e71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:00.192200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 836/1679] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:00.196161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58502f32-f822-47ec-983a-37a40ff3f2a8 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:00.217734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58502f32-f822-47ec-983a-37a40ff3f2a8 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:00.236877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ace3b-4702-4339-830e-117047405c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:00.237675 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ace3b-4702-4339-830e-117047405c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:00.240155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ace3b-4702-4339-830e-117047405c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:00.240581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ace3b-4702-4339-830e-117047405c86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:00.241575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ace3b-4702-4339-830e-117047405c86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:00.241991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ace3b-4702-4339-830e-117047405c86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:00.245504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ace3b-4702-4339-830e-117047405c86 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:00.261198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 844/1680] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:32:00.265731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5865cf5-3e31-4a06-bf6a-2ccf83a4e71b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:00.278518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5865cf5-3e31-4a06-bf6a-2ccf83a4e71b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:00.288637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0e03c-4018-440e-833b-f5fb502927d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:00.289091 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0e03c-4018-440e-833b-f5fb502927d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:00.289482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0e03c-4018-440e-833b-f5fb502927d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:00.289868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0e03c-4018-440e-833b-f5fb502927d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:00.290628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0e03c-4018-440e-833b-f5fb502927d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:00.293596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0e03c-4018-440e-833b-f5fb502927d6 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:00.358527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 837/1681] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:00 2022] POST /identity/v3/projects => generated 363 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:32:00.362769 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ace3b-4702-4339-830e-117047405c86 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:00.382571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ace3b-4702-4339-830e-117047405c86 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:00.503183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0e03c-4018-440e-833b-f5fb502927d6 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:00.507277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3ea0e03c-4018-440e-833b-f5fb502927d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:00.548792 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:00.587512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21559dc3-2cc7-4e06-863a-802c994c33d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:00.588348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21559dc3-2cc7-4e06-863a-802c994c33d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:00.588638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21559dc3-2cc7-4e06-863a-802c994c33d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:00.589076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21559dc3-2cc7-4e06-863a-802c994c33d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:00.591069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21559dc3-2cc7-4e06-863a-802c994c33d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:00.598043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21559dc3-2cc7-4e06-863a-802c994c33d4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:00.649623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 845/1682] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:00 2022] POST /identity/v3/users => generated 282 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:00.654885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21559dc3-2cc7-4e06-863a-802c994c33d4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:00.672339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21559dc3-2cc7-4e06-863a-802c994c33d4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:00.685652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c8481-13de-4a8c-af24-56b55910aa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:00.686283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c8481-13de-4a8c-af24-56b55910aa6c admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:00.686573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c8481-13de-4a8c-af24-56b55910aa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:00.686850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c8481-13de-4a8c-af24-56b55910aa6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:00.687503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c8481-13de-4a8c-af24-56b55910aa6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:00.694108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c8481-13de-4a8c-af24-56b55910aa6c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:00.720061 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 846/1683] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:00.727658 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ddf81-4857-45ad-95c3-b67e87236497 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:00.728015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ddf81-4857-45ad-95c3-b67e87236497 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:00.728302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ddf81-4857-45ad-95c3-b67e87236497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:00.738835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0e03c-4018-440e-833b-f5fb502927d6 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:00.744644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:00.745464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:00.745899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:00.746274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] PATH_INFO: `/v3/projects/e9128b595c6d482fa86897ad44d80fff` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:00.747681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9128b595c6d482fa86897ad44d80fff)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:00.749748 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:00.794358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-dc1ddf81-4857-45ad-95c3-b67e87236497 None None] MFA Rules not processed for user `9b0455ae3ec94c929c940df6a0f6687d`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:00.795514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-dc1ddf81-4857-45ad-95c3-b67e87236497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:00.823429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:00.830066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9128b595c6d482fa86897ad44d80fff', 'request_id': 'req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:00.843080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9128b595c6d482fa86897ad44d80fff', 'request_id': 'req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:00.852808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9128b595c6d482fa86897ad44d80fff', 'request_id': 'req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:00.879578 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] The token cache is being invalidate because project e9128b595c6d482fa86897ad44d80fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:00.881205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:00.883250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:00.890015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 838/1684] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:00 2022] DELETE /identity/v3/projects/e9128b595c6d482fa86897ad44d80fff => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:00.892092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:00.895373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:01.019422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 847/1685] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:01.020084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ddf81-4857-45ad-95c3-b67e87236497 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:01.026006 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-dc1ddf81-4857-45ad-95c3-b67e87236497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:01.062433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ddf81-4857-45ad-95c3-b67e87236497 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:01.080211 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:01.082787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:01.083085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:01.083362 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] PATH_INFO: `/v3/users/462d0ac51bd644d3860a2310f524d034` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:01.086212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ab435-42cb-4cd7-8aac-1daf4ab7e32b admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:01.087109 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=462d0ac51bd644d3860a2310f524d034)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:01.089113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:01.101536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d4a4f-09bb-4c2b-84c5-d53a2c815970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:01.106260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d4a4f-09bb-4c2b-84c5-d53a2c815970 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:01.106551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d4a4f-09bb-4c2b-84c5-d53a2c815970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:01.108420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d4a4f-09bb-4c2b-84c5-d53a2c815970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:01.109307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d4a4f-09bb-4c2b-84c5-d53a2c815970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:01.112648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d4a4f-09bb-4c2b-84c5-d53a2c815970 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:01.150897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 839/1686] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:01.153363 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2716f6-ce7f-4233-8078-15352effff8f None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:01.153665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2716f6-ce7f-4233-8078-15352effff8f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:01.153948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2716f6-ce7f-4233-8078-15352effff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:01.222723 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ee2716f6-ce7f-4233-8078-15352effff8f None None] MFA Rules not processed for user `e09fd34737af41dc972d806dc3b0705e`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:01.229471 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ee2716f6-ce7f-4233-8078-15352effff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:01.235288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '462d0ac51bd644d3860a2310f524d034'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:01.238331 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '462d0ac51bd644d3860a2310f524d034'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:01.240369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '462d0ac51bd644d3860a2310f524d034'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:01.298787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '462d0ac51bd644d3860a2310f524d034'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:01.316864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '462d0ac51bd644d3860a2310f524d034'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:01.350698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 848/1687] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:01 2022] DELETE /identity/v3/users/462d0ac51bd644d3860a2310f524d034 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:01.351607 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:01.353338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:01.422786 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 840/1688] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:01.424328 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2716f6-ce7f-4233-8078-15352effff8f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:01.428091 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ee2716f6-ce7f-4233-8078-15352effff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:01.468441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2716f6-ce7f-4233-8078-15352effff8f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:01.470760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:01.613011 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a123b81-300f-4bc6-86c0-0af5c0a46538 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:01.642405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed925-a4a8-4b79-a294-7790bfbb25f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:01.644016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed925-a4a8-4b79-a294-7790bfbb25f3 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:01.644521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed925-a4a8-4b79-a294-7790bfbb25f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:01.645039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed925-a4a8-4b79-a294-7790bfbb25f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:01.649501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a4fed925-a4a8-4b79-a294-7790bfbb25f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:01.671666 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3ce4d-5609-42fe-a2fb-e8125c3ff482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:01.672541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3ce4d-5609-42fe-a2fb-e8125c3ff482 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:01.673041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3ce4d-5609-42fe-a2fb-e8125c3ff482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:01.673511 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3ce4d-5609-42fe-a2fb-e8125c3ff482 admin admin] PATH_INFO: `/v3/projects/fcd3d475eebf4f21988e04d3b6e2e0e3/users/f8d400134bed49288b8cda35ce1ebe40/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:01.711948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3ce4d-5609-42fe-a2fb-e8125c3ff482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd3d475eebf4f21988e04d3b6e2e0e3, user_id=f8d400134bed49288b8cda35ce1ebe40, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:01.714148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3ce4d-5609-42fe-a2fb-e8125c3ff482 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:01.720574 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:01.740336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 841/1689] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:01 2022] PUT /identity/v3/projects/fcd3d475eebf4f21988e04d3b6e2e0e3/users/f8d400134bed49288b8cda35ce1ebe40/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:01.752768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab56ec-3315-454a-919f-27179ce6f9d8 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:01.753347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab56ec-3315-454a-919f-27179ce6f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:01.753347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab56ec-3315-454a-919f-27179ce6f9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:01.837823 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-e8ab56ec-3315-454a-919f-27179ce6f9d8 None None] MFA Rules not processed for user `f8d400134bed49288b8cda35ce1ebe40`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:01.845566 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e8ab56ec-3315-454a-919f-27179ce6f9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:01.963776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fed925-a4a8-4b79-a294-7790bfbb25f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:01.971316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fed925-a4a8-4b79-a294-7790bfbb25f3 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:01.997263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 849/1690] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:01.998620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83694b04-38d4-430d-992c-94a892a0fe07 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:02.003067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83694b04-38d4-430d-992c-94a892a0fe07 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:02.003395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83694b04-38d4-430d-992c-94a892a0fe07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:02.037092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 842/1691] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:02.041203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab56ec-3315-454a-919f-27179ce6f9d8 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:02.089879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-83694b04-38d4-430d-992c-94a892a0fe07 None None] MFA Rules not processed for user `7936bf23a031418b8ce304aa3bbcf8a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:02.092016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-83694b04-38d4-430d-992c-94a892a0fe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:02.178274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab56ec-3315-454a-919f-27179ce6f9d8 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:02.185819 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef13da8-684a-44d2-b5c6-8ebe429a2613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:02.186474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13da8-684a-44d2-b5c6-8ebe429a2613 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:02.186759 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13da8-684a-44d2-b5c6-8ebe429a2613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:02.187037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13da8-684a-44d2-b5c6-8ebe429a2613 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:02.187700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef13da8-684a-44d2-b5c6-8ebe429a2613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:02.189761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef13da8-684a-44d2-b5c6-8ebe429a2613 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:02.227661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 843/1692] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:02 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:32:02.232829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef13da8-684a-44d2-b5c6-8ebe429a2613 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:02.341894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 850/1693] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:02.354236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c4cef-4447-400c-a0d4-ba16b2701c8a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:02.354530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c4cef-4447-400c-a0d4-ba16b2701c8a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:02.354802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c4cef-4447-400c-a0d4-ba16b2701c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:02.417271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef13da8-684a-44d2-b5c6-8ebe429a2613 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:02.427822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-fe9c4cef-4447-400c-a0d4-ba16b2701c8a None None] MFA Rules not processed for user `7936bf23a031418b8ce304aa3bbcf8a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:02.432047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fe9c4cef-4447-400c-a0d4-ba16b2701c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:02.434077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb9d64-5ebc-458d-8c59-79b0da1e11e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:02.434720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9d64-5ebc-458d-8c59-79b0da1e11e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:02.434998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9d64-5ebc-458d-8c59-79b0da1e11e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:02.435266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9d64-5ebc-458d-8c59-79b0da1e11e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:02.435905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb9d64-5ebc-458d-8c59-79b0da1e11e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:02.437885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb9d64-5ebc-458d-8c59-79b0da1e11e8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:02.499787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:02.505723 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 844/1694] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:02 2022] POST /identity/v3/users => generated 289 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:02.509690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb9d64-5ebc-458d-8c59-79b0da1e11e8 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:02.560693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb9d64-5ebc-458d-8c59-79b0da1e11e8 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:02.570459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250f858-31ed-49c1-b004-dfeb7bd35799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:02.571369 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250f858-31ed-49c1-b004-dfeb7bd35799 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:02.571814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250f858-31ed-49c1-b004-dfeb7bd35799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:02.572232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250f858-31ed-49c1-b004-dfeb7bd35799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:02.573059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250f858-31ed-49c1-b004-dfeb7bd35799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:02.578982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250f858-31ed-49c1-b004-dfeb7bd35799 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:02.615603 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:02.627510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 851/1695] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:02.634603 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 845/1696] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:02.635906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c4cef-4447-400c-a0d4-ba16b2701c8a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:02.639561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250f858-31ed-49c1-b004-dfeb7bd35799 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:02.659648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c4cef-4447-400c-a0d4-ba16b2701c8a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:02.683750 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ad211-f60c-403e-9cdd-cda9521602d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:02.684450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ad211-f60c-403e-9cdd-cda9521602d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:02.684729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ad211-f60c-403e-9cdd-cda9521602d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:02.685023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ad211-f60c-403e-9cdd-cda9521602d5 admin admin] PATH_INFO: `/v3/projects/baf4dfe69cb0461fa060c2662766648b/users/f93f88ceddc940c38170e9067d431e4f/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:02.741615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ad211-f60c-403e-9cdd-cda9521602d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf4dfe69cb0461fa060c2662766648b, user_id=f93f88ceddc940c38170e9067d431e4f, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:02.744603 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ad211-f60c-403e-9cdd-cda9521602d5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:02.790466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 852/1697] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:02 2022] PUT /identity/v3/projects/baf4dfe69cb0461fa060c2662766648b/users/f93f88ceddc940c38170e9067d431e4f/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:02.796588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ad211-f60c-403e-9cdd-cda9521602d5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:02.834176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250f858-31ed-49c1-b004-dfeb7bd35799 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:02.954388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ad211-f60c-403e-9cdd-cda9521602d5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:02.967059 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f859364-ed48-413b-b46a-c9e636e1b51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:02.967703 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f859364-ed48-413b-b46a-c9e636e1b51c admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:02.967985 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f859364-ed48-413b-b46a-c9e636e1b51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:02.968673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f859364-ed48-413b-b46a-c9e636e1b51c admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:02.969333 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f859364-ed48-413b-b46a-c9e636e1b51c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:02.971038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f859364-ed48-413b-b46a-c9e636e1b51c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:03.004626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 853/1698] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:03.008426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f859364-ed48-413b-b46a-c9e636e1b51c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:03.033415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f859364-ed48-413b-b46a-c9e636e1b51c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:03.044178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe9519-05ee-445e-b85b-2a8c9cf66ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:03.045421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe9519-05ee-445e-b85b-2a8c9cf66ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:03.045848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe9519-05ee-445e-b85b-2a8c9cf66ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:03.046279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe9519-05ee-445e-b85b-2a8c9cf66ec5 admin admin] PATH_INFO: `/v3/projects/baf4dfe69cb0461fa060c2662766648b/users/f93f88ceddc940c38170e9067d431e4f/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:03.052270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe9519-05ee-445e-b85b-2a8c9cf66ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf4dfe69cb0461fa060c2662766648b, user_id=f93f88ceddc940c38170e9067d431e4f, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:03.056432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebbce8-6893-4b07-8204-3be7bc06ab7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:03.057106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebbce8-6893-4b07-8204-3be7bc06ab7b service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:03.057440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebbce8-6893-4b07-8204-3be7bc06ab7b service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:03.057718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebbce8-6893-4b07-8204-3be7bc06ab7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:03.065192 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe9519-05ee-445e-b85b-2a8c9cf66ec5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:03.084709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 854/1699] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:03 2022] PUT /identity/v3/projects/baf4dfe69cb0461fa060c2662766648b/users/f93f88ceddc940c38170e9067d431e4f/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:03.099094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bfb76-52cf-4e44-aac5-7e09753a3e74 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:03.102583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bfb76-52cf-4e44-aac5-7e09753a3e74 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:03.103039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bfb76-52cf-4e44-aac5-7e09753a3e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:03.162577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7c9bfb76-52cf-4e44-aac5-7e09753a3e74 None None] MFA Rules not processed for user `f93f88ceddc940c38170e9067d431e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:03.164225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7c9bfb76-52cf-4e44-aac5-7e09753a3e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:03.218663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebbce8-6893-4b07-8204-3be7bc06ab7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:03.220299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebbce8-6893-4b07-8204-3be7bc06ab7b service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:03.255222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 846/1700] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:03.283435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 855/1701] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:03.291612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebbce8-6893-4b07-8204-3be7bc06ab7b service nova] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:03.300177 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0eebbce8-6893-4b07-8204-3be7bc06ab7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:03.368942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:03.449929 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebbce8-6893-4b07-8204-3be7bc06ab7b service nova] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:03.558094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544d118-38fc-4069-8ec5-b96a6eec2714 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:03.558481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544d118-38fc-4069-8ec5-b96a6eec2714 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:03.558720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544d118-38fc-4069-8ec5-b96a6eec2714 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:03.560198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 856/1702] 10.222.0.55 () {54 vars in 912 bytes} [Fri Aug 19 16:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:32:03.569695 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe396c-af8c-4119-a742-1a1cf5993043 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:03.569878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f4ab0-0888-4bb9-9f29-4ac879f49e69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:03.570008 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe396c-af8c-4119-a742-1a1cf5993043 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:03.570242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe396c-af8c-4119-a742-1a1cf5993043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:03.570480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f4ab0-0888-4bb9-9f29-4ac879f49e69 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:03.570772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f4ab0-0888-4bb9-9f29-4ac879f49e69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:03.571047 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f4ab0-0888-4bb9-9f29-4ac879f49e69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:03.594204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f4ab0-0888-4bb9-9f29-4ac879f49e69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:03.595689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f4ab0-0888-4bb9-9f29-4ac879f49e69 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:03.626491 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 847/1703] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:03.670302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9afe396c-af8c-4119-a742-1a1cf5993043 None None] MFA Rules not processed for user `9e0b7efc488a42ffab8ba7e2d101ced4`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:03.671679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9afe396c-af8c-4119-a742-1a1cf5993043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:03.734773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:03.783277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f4ab0-0888-4bb9-9f29-4ac879f49e69 service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:03.798180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f4ab0-0888-4bb9-9f29-4ac879f49e69 service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:03.809089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf607960-4afe-42b2-baaf-180cb46781ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:03.809726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf607960-4afe-42b2-baaf-180cb46781ae service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:03.810006 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf607960-4afe-42b2-baaf-180cb46781ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:03.811938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf607960-4afe-42b2-baaf-180cb46781ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:03.814315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 857/1704] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:32:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:03.861623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe396c-af8c-4119-a742-1a1cf5993043 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:03.863119 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9afe396c-af8c-4119-a742-1a1cf5993043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:03.881836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe396c-af8c-4119-a742-1a1cf5993043 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:03.892545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090616d-11b5-4cc8-81ab-c20781392b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:03.893139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090616d-11b5-4cc8-81ab-c20781392b46 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:03.893421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090616d-11b5-4cc8-81ab-c20781392b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:03.893696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090616d-11b5-4cc8-81ab-c20781392b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:03.910012 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090616d-11b5-4cc8-81ab-c20781392b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:03.914158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090616d-11b5-4cc8-81ab-c20781392b46 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:03.939468 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 858/1705] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:03.972482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf607960-4afe-42b2-baaf-180cb46781ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:03.974850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf607960-4afe-42b2-baaf-180cb46781ae service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:03.995793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 848/1706] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:04.409155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090616d-11b5-4cc8-81ab-c20781392b46 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:04.410957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b090616d-11b5-4cc8-81ab-c20781392b46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:04.578168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621dbdb-fa90-45d0-9c49-75e3b753306b None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:04.579132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621dbdb-fa90-45d0-9c49-75e3b753306b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:04.579644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621dbdb-fa90-45d0-9c49-75e3b753306b None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:04.582220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 849/1707] 10.222.0.55 () {54 vars in 912 bytes} [Fri Aug 19 16:32:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:32:04.588333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f4b83-a95b-443a-951d-2121007bd7a7 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:04.589030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f4b83-a95b-443a-951d-2121007bd7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:04.590253 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f4b83-a95b-443a-951d-2121007bd7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:04.635905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090616d-11b5-4cc8-81ab-c20781392b46 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:04.645282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-8f7f4b83-a95b-443a-951d-2121007bd7a7 None None] MFA Rules not processed for user `9e0b7efc488a42ffab8ba7e2d101ced4`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:04.646669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8f7f4b83-a95b-443a-951d-2121007bd7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:04.651285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:04.652287 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:04.652853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:04.653130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] PATH_INFO: `/v3/projects/6e5563fb60c2429d86350b30d5f7ff7b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:04.657156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 850/1708] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:32:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:04.658593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e5563fb60c2429d86350b30d5f7ff7b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:04.658682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27321c4-1c49-469b-9ecd-8356760ea00e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:04.658980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27321c4-1c49-469b-9ecd-8356760ea00e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:04.659364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27321c4-1c49-469b-9ecd-8356760ea00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:04.664953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:04.707726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f27321c4-1c49-469b-9ecd-8356760ea00e None None] MFA Rules not processed for user `d55dc52ab80642da90f05e74092ffdf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:04.710017 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f27321c4-1c49-469b-9ecd-8356760ea00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:04.797506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e5563fb60c2429d86350b30d5f7ff7b', 'request_id': 'req-9632732a-67b9-4cae-9838-7e609e8714f9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:04.811491 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e5563fb60c2429d86350b30d5f7ff7b', 'request_id': 'req-9632732a-67b9-4cae-9838-7e609e8714f9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:04.827373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e5563fb60c2429d86350b30d5f7ff7b', 'request_id': 'req-9632732a-67b9-4cae-9838-7e609e8714f9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:04.864047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] The token cache is being invalidate because project 6e5563fb60c2429d86350b30d5f7ff7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:04.864415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:04.865755 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:04.869160 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 859/1709] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:04 2022] DELETE /identity/v3/projects/6e5563fb60c2429d86350b30d5f7ff7b => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:04.871330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:04.873524 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:04.988957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 851/1710] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:04.991259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829d054-fbe0-4ba0-affd-0504364aa398 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:04.992441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829d054-fbe0-4ba0-affd-0504364aa398 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:04.992908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829d054-fbe0-4ba0-affd-0504364aa398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:05.022564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:05.033441 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632732a-67b9-4cae-9838-7e609e8714f9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:05.042076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca70cc-17ee-424d-a04d-c2882cd46f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:05.043457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca70cc-17ee-424d-a04d-c2882cd46f3d service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:05.043997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca70cc-17ee-424d-a04d-c2882cd46f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:05.044348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca70cc-17ee-424d-a04d-c2882cd46f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:05.045933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-9829d054-fbe0-4ba0-affd-0504364aa398 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:05.047357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a5ca70cc-17ee-424d-a04d-c2882cd46f3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:05.050502 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9829d054-fbe0-4ba0-affd-0504364aa398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:05.082733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca70cc-17ee-424d-a04d-c2882cd46f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:05.084548 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca70cc-17ee-424d-a04d-c2882cd46f3d service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:05.094979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:05.110228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 860/1711] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:05.112771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5af26c-136c-48b3-b209-44164f3d7aee None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:05.113358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5af26c-136c-48b3-b209-44164f3d7aee None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:05.114771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5af26c-136c-48b3-b209-44164f3d7aee None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:05.123566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 861/1712] 10.222.0.55 () {54 vars in 961 bytes} [Fri Aug 19 16:32:05 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:32:05.127043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855dd8c-e197-4a5c-8ccc-cbc26dbed929 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:05.127968 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855dd8c-e197-4a5c-8ccc-cbc26dbed929 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:05.128350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855dd8c-e197-4a5c-8ccc-cbc26dbed929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:05.171912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d855dd8c-e197-4a5c-8ccc-cbc26dbed929 None None] MFA Rules not processed for user `1dcbe2b128a94b9abbf3a9b25db1e845`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:05.174929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d855dd8c-e197-4a5c-8ccc-cbc26dbed929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:05.254081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 852/1713] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:05.264077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f746e95-27c5-4c90-b653-2505de8e4b73 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:05.265475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f746e95-27c5-4c90-b653-2505de8e4b73 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:05.265905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f746e95-27c5-4c90-b653-2505de8e4b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:05.321821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-8f746e95-27c5-4c90-b653-2505de8e4b73 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:05.325746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8f746e95-27c5-4c90-b653-2505de8e4b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:05.347153 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 853/1714] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:05.351245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f746e95-27c5-4c90-b653-2505de8e4b73 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:05.374445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f746e95-27c5-4c90-b653-2505de8e4b73 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:05.380498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 862/1715] 10.222.0.55 () {58 vars in 1058 bytes} [Fri Aug 19 16:32:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:05.391933 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855dd8c-e197-4a5c-8ccc-cbc26dbed929 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:05.398034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74798c-3261-46fb-8271-e0e217be2907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:05.399004 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74798c-3261-46fb-8271-e0e217be2907 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:05.399477 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74798c-3261-46fb-8271-e0e217be2907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:05.400035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74798c-3261-46fb-8271-e0e217be2907 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:05.400902 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74798c-3261-46fb-8271-e0e217be2907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:05.401435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74798c-3261-46fb-8271-e0e217be2907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:05.414127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74798c-3261-46fb-8271-e0e217be2907 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:05.450921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855dd8c-e197-4a5c-8ccc-cbc26dbed929 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:05.470448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f58150-b8ca-4c48-81b0-91277d1a08c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:05.472946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f58150-b8ca-4c48-81b0-91277d1a08c4 service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:05.474019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f58150-b8ca-4c48-81b0-91277d1a08c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:05.475013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f58150-b8ca-4c48-81b0-91277d1a08c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:05.482873 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-54f58150-b8ca-4c48-81b0-91277d1a08c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:05.485224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 854/1716] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:32:05.489088 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74798c-3261-46fb-8271-e0e217be2907 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:05.518266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f58150-b8ca-4c48-81b0-91277d1a08c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:05.520322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f58150-b8ca-4c48-81b0-91277d1a08c4 service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:05.537283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74798c-3261-46fb-8271-e0e217be2907 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:05.546503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 863/1717] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:05.547261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f58150-b8ca-4c48-81b0-91277d1a08c4 service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:05.548374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31adfaa-a3e8-4c04-83a1-ce04bd099054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:05.549611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-54f58150-b8ca-4c48-81b0-91277d1a08c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:05.551512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31adfaa-a3e8-4c04-83a1-ce04bd099054 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:05.551977 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31adfaa-a3e8-4c04-83a1-ce04bd099054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:05.552543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31adfaa-a3e8-4c04-83a1-ce04bd099054 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:05.553430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31adfaa-a3e8-4c04-83a1-ce04bd099054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:05.564069 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31adfaa-a3e8-4c04-83a1-ce04bd099054 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:05.634901 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 855/1718] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:05 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:05.642487 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31adfaa-a3e8-4c04-83a1-ce04bd099054 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:05.837202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31adfaa-a3e8-4c04-83a1-ce04bd099054 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:05.847917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515498d-01af-4903-bc5d-de0d26f90f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:05.849375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515498d-01af-4903-bc5d-de0d26f90f27 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:05.853569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515498d-01af-4903-bc5d-de0d26f90f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:05.853848 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515498d-01af-4903-bc5d-de0d26f90f27 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:05.854515 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515498d-01af-4903-bc5d-de0d26f90f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:05.873202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515498d-01af-4903-bc5d-de0d26f90f27 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:05.897643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f58150-b8ca-4c48-81b0-91277d1a08c4 service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:05.921695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 856/1719] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:05 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:05.923590 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515498d-01af-4903-bc5d-de0d26f90f27 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:05.979373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515498d-01af-4903-bc5d-de0d26f90f27 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:06.001147 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3305c45-89fd-4684-acd9-8a160a3e6bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:06.004398 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3305c45-89fd-4684-acd9-8a160a3e6bfc admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:06.005276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3305c45-89fd-4684-acd9-8a160a3e6bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:06.007644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3305c45-89fd-4684-acd9-8a160a3e6bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:06.009081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3305c45-89fd-4684-acd9-8a160a3e6bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:06.010920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3305c45-89fd-4684-acd9-8a160a3e6bfc admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:06.042817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 857/1720] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:06.045716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3305c45-89fd-4684-acd9-8a160a3e6bfc admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:06.071696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3305c45-89fd-4684-acd9-8a160a3e6bfc admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:06.085002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278667f7-c526-4f0e-b23d-22a8a19af95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:06.086774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278667f7-c526-4f0e-b23d-22a8a19af95d admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:06.087213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278667f7-c526-4f0e-b23d-22a8a19af95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:06.087757 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278667f7-c526-4f0e-b23d-22a8a19af95d admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:06.090539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278667f7-c526-4f0e-b23d-22a8a19af95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:06.095799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278667f7-c526-4f0e-b23d-22a8a19af95d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:06.124939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 858/1721] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:06.129318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278667f7-c526-4f0e-b23d-22a8a19af95d admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:06.158978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278667f7-c526-4f0e-b23d-22a8a19af95d admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:06.172728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4436b-902b-411b-beb5-684ca9459d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:06.175632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4436b-902b-411b-beb5-684ca9459d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:06.176070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4436b-902b-411b-beb5-684ca9459d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:06.177922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4436b-902b-411b-beb5-684ca9459d3d admin admin] PATH_INFO: `/v3/projects/a69355df1a0d4be69b77d7688a01e325/users/7306a5753d574f3aa259a5ee2bb8c210/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:06.199648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1f54d-3d0d-4bee-8d4c-4bb95dc2646b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:06.203220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1f54d-3d0d-4bee-8d4c-4bb95dc2646b service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:06.203546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1f54d-3d0d-4bee-8d4c-4bb95dc2646b service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:06.203835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1f54d-3d0d-4bee-8d4c-4bb95dc2646b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:06.247443 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4436b-902b-411b-beb5-684ca9459d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a69355df1a0d4be69b77d7688a01e325, user_id=7306a5753d574f3aa259a5ee2bb8c210, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:06.256216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4436b-902b-411b-beb5-684ca9459d3d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:06.262801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:06.318370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 859/1722] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:06 2022] PUT /identity/v3/projects/a69355df1a0d4be69b77d7688a01e325/users/7306a5753d574f3aa259a5ee2bb8c210/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:06.329277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae2cad-4a97-4d86-ae86-494b11ee4028 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:06.329582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae2cad-4a97-4d86-ae86-494b11ee4028 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:06.329833 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae2cad-4a97-4d86-ae86-494b11ee4028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:06.378708 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:06.417693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-60ae2cad-4a97-4d86-ae86-494b11ee4028 None None] MFA Rules not processed for user `7306a5753d574f3aa259a5ee2bb8c210`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:06.419113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-60ae2cad-4a97-4d86-ae86-494b11ee4028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:06.462786 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1f54d-3d0d-4bee-8d4c-4bb95dc2646b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:06.465808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1f54d-3d0d-4bee-8d4c-4bb95dc2646b service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:06.484213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 864/1723] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 937 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:06.797053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1f54d-3d0d-4bee-8d4c-4bb95dc2646b service nova] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:06.799022 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d8f1f54d-3d0d-4bee-8d4c-4bb95dc2646b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:06.809369 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 860/1724] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:06.848184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7bb5f-8423-42d2-888e-7fa4bb73ab4b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:06.848480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7bb5f-8423-42d2-888e-7fa4bb73ab4b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:06.848780 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7bb5f-8423-42d2-888e-7fa4bb73ab4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:06.936600 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-48c7bb5f-8423-42d2-888e-7fa4bb73ab4b None None] MFA Rules not processed for user `7936bf23a031418b8ce304aa3bbcf8a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:06.945143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-48c7bb5f-8423-42d2-888e-7fa4bb73ab4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:07.074489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1f54d-3d0d-4bee-8d4c-4bb95dc2646b service nova] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:07.156789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 861/1725] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:07.295064 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16a481-5b2f-436f-8330-df7cc5370a5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:07.296550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16a481-5b2f-436f-8330-df7cc5370a5e service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:07.297922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16a481-5b2f-436f-8330-df7cc5370a5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:07.300311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16a481-5b2f-436f-8330-df7cc5370a5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:07.320989 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16a481-5b2f-436f-8330-df7cc5370a5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:07.325156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16a481-5b2f-436f-8330-df7cc5370a5e service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:07.369478 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 865/1726] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:07.491378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7bb5f-8423-42d2-888e-7fa4bb73ab4b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:07.493635 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-48c7bb5f-8423-42d2-888e-7fa4bb73ab4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:07.719523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7bb5f-8423-42d2-888e-7fa4bb73ab4b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:07.731485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:07.733249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:07.737794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:07.738190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] PATH_INFO: `/v3/users/21be6a5796114b218cd028f57a437cdd` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:07.740133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21be6a5796114b218cd028f57a437cdd)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:07.742585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:07.902055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:07.906796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21be6a5796114b218cd028f57a437cdd'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:07.910165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21be6a5796114b218cd028f57a437cdd'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:07.914359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21be6a5796114b218cd028f57a437cdd'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:07.945626 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21be6a5796114b218cd028f57a437cdd'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:07.994814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21be6a5796114b218cd028f57a437cdd'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:08.036337 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 862/1727] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:07 2022] DELETE /identity/v3/users/21be6a5796114b218cd028f57a437cdd => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:08.041466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c624be-e40a-4362-a252-ea32f7598382 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:08.042430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c624be-e40a-4362-a252-ea32f7598382 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:08.042430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c624be-e40a-4362-a252-ea32f7598382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:08.144477 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-27c624be-e40a-4362-a252-ea32f7598382 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:08.156622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-27c624be-e40a-4362-a252-ea32f7598382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:08.437438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 866/1728] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:08.534294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:08.536380 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:08.764196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb38fc3-566d-4c23-8844-4ea79dd09a30 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:08.778764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8b7d2-d789-4024-a0bf-3eae01f30327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:08.779470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8b7d2-d789-4024-a0bf-3eae01f30327 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:08.783178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8b7d2-d789-4024-a0bf-3eae01f30327 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:08.783724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8b7d2-d789-4024-a0bf-3eae01f30327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:08.829668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f77a2-cb5d-41d1-932f-de3fc9f7dee4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:08.829958 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f77a2-cb5d-41d1-932f-de3fc9f7dee4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:08.830239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f77a2-cb5d-41d1-932f-de3fc9f7dee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:08.831556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8b7d2-d789-4024-a0bf-3eae01f30327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:08.833087 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8b7d2-d789-4024-a0bf-3eae01f30327 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:08.883672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 863/1729] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:08.926483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-564f77a2-cb5d-41d1-932f-de3fc9f7dee4 None None] MFA Rules not processed for user `8d20b306fc68435baa0da6f09c915157`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:08.938925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-564f77a2-cb5d-41d1-932f-de3fc9f7dee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:08.967761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:09.186769 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 867/1730] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:09.200428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65079b-0f6e-47ac-bb63-c2af66af2217 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:09.201042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65079b-0f6e-47ac-bb63-c2af66af2217 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:09.201042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65079b-0f6e-47ac-bb63-c2af66af2217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:09.264891 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-1d65079b-0f6e-47ac-bb63-c2af66af2217 None None] MFA Rules not processed for user `8d20b306fc68435baa0da6f09c915157`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:09.267032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-1d65079b-0f6e-47ac-bb63-c2af66af2217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:09.281401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 864/1731] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:09.288318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f77a2-cb5d-41d1-932f-de3fc9f7dee4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:09.289880 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-564f77a2-cb5d-41d1-932f-de3fc9f7dee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:09.501931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f77a2-cb5d-41d1-932f-de3fc9f7dee4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:09.528959 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd1114-a738-4b43-9939-fdfc7693cd96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:09.531577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd1114-a738-4b43-9939-fdfc7693cd96 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:09.531877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd1114-a738-4b43-9939-fdfc7693cd96 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:09.532361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd1114-a738-4b43-9939-fdfc7693cd96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:09.563328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd1114-a738-4b43-9939-fdfc7693cd96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:09.565008 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd1114-a738-4b43-9939-fdfc7693cd96 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:09.610415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 868/1732] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:10.493079 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65079b-0f6e-47ac-bb63-c2af66af2217 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:10.498234 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-1d65079b-0f6e-47ac-bb63-c2af66af2217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:10.552049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65079b-0f6e-47ac-bb63-c2af66af2217 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:10.561518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5e095-a3f6-45c2-8386-283d958c141b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:10.562860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5e095-a3f6-45c2-8386-283d958c141b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:10.564488 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5e095-a3f6-45c2-8386-283d958c141b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:10.564878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5e095-a3f6-45c2-8386-283d958c141b admin admin] PATH_INFO: `/v3/projects/e0c2404dc80e4cf9ad6b109b3aabb9d5` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:10.566995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5e095-a3f6-45c2-8386-283d958c141b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0c2404dc80e4cf9ad6b109b3aabb9d5)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:10.572330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5e095-a3f6-45c2-8386-283d958c141b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:10.700552 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2db5e095-a3f6-45c2-8386-283d958c141b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0c2404dc80e4cf9ad6b109b3aabb9d5', 'request_id': 'req-2db5e095-a3f6-45c2-8386-283d958c141b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:10.718386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2db5e095-a3f6-45c2-8386-283d958c141b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0c2404dc80e4cf9ad6b109b3aabb9d5', 'request_id': 'req-2db5e095-a3f6-45c2-8386-283d958c141b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:10.747717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2db5e095-a3f6-45c2-8386-283d958c141b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0c2404dc80e4cf9ad6b109b3aabb9d5', 'request_id': 'req-2db5e095-a3f6-45c2-8386-283d958c141b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:10.775079 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2db5e095-a3f6-45c2-8386-283d958c141b admin admin] The token cache is being invalidate because project e0c2404dc80e4cf9ad6b109b3aabb9d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:10.775635 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2db5e095-a3f6-45c2-8386-283d958c141b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:10.777911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2db5e095-a3f6-45c2-8386-283d958c141b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:10.780052 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 865/1733] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:10 2022] DELETE /identity/v3/projects/e0c2404dc80e4cf9ad6b109b3aabb9d5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:10.785303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd1114-a738-4b43-9939-fdfc7693cd96 service nova] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:10.817714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd1114-a738-4b43-9939-fdfc7693cd96 service nova] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:10.941649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:10.942427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:10.942717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:10.942994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] PATH_INFO: `/v3/users/0418a67fb0114b809b01ee6fa0d409ff` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:10.945550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0418a67fb0114b809b01ee6fa0d409ff)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:10.952496 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:11.029988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa998d3-a78f-40a3-bb81-86714126f129 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:11.030276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa998d3-a78f-40a3-bb81-86714126f129 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:11.030559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa998d3-a78f-40a3-bb81-86714126f129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:11.060457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:11.081967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-efa998d3-a78f-40a3-bb81-86714126f129 None None] MFA Rules not processed for user `9b0455ae3ec94c929c940df6a0f6687d`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:11.083222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-efa998d3-a78f-40a3-bb81-86714126f129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:11.090255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0418a67fb0114b809b01ee6fa0d409ff'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:11.091408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0418a67fb0114b809b01ee6fa0d409ff'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:11.092292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0418a67fb0114b809b01ee6fa0d409ff'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:11.137662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0418a67fb0114b809b01ee6fa0d409ff'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:11.140104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:11.153806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0418a67fb0114b809b01ee6fa0d409ff'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:11.187061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 866/1734] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:11.196197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 869/1735] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:10 2022] DELETE /identity/v3/users/0418a67fb0114b809b01ee6fa0d409ff => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:11.871989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514e426-b47a-4ef0-9b93-5fcd48087939 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:11.872992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514e426-b47a-4ef0-9b93-5fcd48087939 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:11.873550 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514e426-b47a-4ef0-9b93-5fcd48087939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:11.909979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f514e426-b47a-4ef0-9b93-5fcd48087939 None None] MFA Rules not processed for user `d55dc52ab80642da90f05e74092ffdf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:11.911139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f514e426-b47a-4ef0-9b93-5fcd48087939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:12.063730 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 867/1736] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:12.112875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:12.114569 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:12.334452 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c0a2f-0064-499f-bb53-b4d8123fb525 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:12.367806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e87e3-8e66-4b06-9d99-31054fbd9bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:12.370663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e87e3-8e66-4b06-9d99-31054fbd9bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:12.372618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e87e3-8e66-4b06-9d99-31054fbd9bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:12.373002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e87e3-8e66-4b06-9d99-31054fbd9bd7 admin admin] PATH_INFO: `/v3/projects/ad847618ee09484a96cda3bd83da358f` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:12.377179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e87e3-8e66-4b06-9d99-31054fbd9bd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad847618ee09484a96cda3bd83da358f)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:12.383598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e87e3-8e66-4b06-9d99-31054fbd9bd7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:12.549141 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-852e87e3-8e66-4b06-9d99-31054fbd9bd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad847618ee09484a96cda3bd83da358f', 'request_id': 'req-852e87e3-8e66-4b06-9d99-31054fbd9bd7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:12.556554 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-852e87e3-8e66-4b06-9d99-31054fbd9bd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad847618ee09484a96cda3bd83da358f', 'request_id': 'req-852e87e3-8e66-4b06-9d99-31054fbd9bd7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:12.572154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-852e87e3-8e66-4b06-9d99-31054fbd9bd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad847618ee09484a96cda3bd83da358f', 'request_id': 'req-852e87e3-8e66-4b06-9d99-31054fbd9bd7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:12.607390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-852e87e3-8e66-4b06-9d99-31054fbd9bd7 admin admin] The token cache is being invalidate because project ad847618ee09484a96cda3bd83da358f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:12.609605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-852e87e3-8e66-4b06-9d99-31054fbd9bd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:12.610575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-852e87e3-8e66-4b06-9d99-31054fbd9bd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:12.616388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 870/1737] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:12 2022] DELETE /identity/v3/projects/ad847618ee09484a96cda3bd83da358f => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:12.623193 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56128793-7001-411a-b559-58f48c69b5b9 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:12.623546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56128793-7001-411a-b559-58f48c69b5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:12.624009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56128793-7001-411a-b559-58f48c69b5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:12.691334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-56128793-7001-411a-b559-58f48c69b5b9 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:12.694086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-56128793-7001-411a-b559-58f48c69b5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:12.833255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 868/1738] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:12.842623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc2f16-4d41-4a39-8a43-52abb31ef75e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:12.846949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc2f16-4d41-4a39-8a43-52abb31ef75e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:12.849737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc2f16-4d41-4a39-8a43-52abb31ef75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:12.891613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c3bc2f16-4d41-4a39-8a43-52abb31ef75e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:12.892647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c3bc2f16-4d41-4a39-8a43-52abb31ef75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:12.901405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 871/1739] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:12.906596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56128793-7001-411a-b559-58f48c69b5b9 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:12.931617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56128793-7001-411a-b559-58f48c69b5b9 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:12.944685 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc358e93-76e7-490b-8c72-d87e7b2a393c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:12.945389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc358e93-76e7-490b-8c72-d87e7b2a393c admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:12.945671 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc358e93-76e7-490b-8c72-d87e7b2a393c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:12.945945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc358e93-76e7-490b-8c72-d87e7b2a393c admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:12.946606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc358e93-76e7-490b-8c72-d87e7b2a393c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:12.946909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc358e93-76e7-490b-8c72-d87e7b2a393c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:12.949297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc358e93-76e7-490b-8c72-d87e7b2a393c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:12.970516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 869/1740] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:32:12.989210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc2f16-4d41-4a39-8a43-52abb31ef75e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:13.010399 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc2f16-4d41-4a39-8a43-52abb31ef75e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:13.020789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32f348-4ab2-40c9-8276-a31514ff44ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:13.022583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32f348-4ab2-40c9-8276-a31514ff44ba admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:13.023391 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32f348-4ab2-40c9-8276-a31514ff44ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:13.024124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32f348-4ab2-40c9-8276-a31514ff44ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:13.026182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32f348-4ab2-40c9-8276-a31514ff44ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:13.030886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32f348-4ab2-40c9-8276-a31514ff44ba admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:13.120829 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 872/1741] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:12 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:13.125900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc358e93-76e7-490b-8c72-d87e7b2a393c admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:13.142616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc358e93-76e7-490b-8c72-d87e7b2a393c admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:13.318096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff86bf-b9c2-474a-ba7e-dbcb31178cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:13.318747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff86bf-b9c2-474a-ba7e-dbcb31178cbf admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:13.319025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff86bf-b9c2-474a-ba7e-dbcb31178cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:13.319293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff86bf-b9c2-474a-ba7e-dbcb31178cbf admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:13.319927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff86bf-b9c2-474a-ba7e-dbcb31178cbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:13.322078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff86bf-b9c2-474a-ba7e-dbcb31178cbf admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:13.360941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 870/1742] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:13 2022] POST /identity/v3/users => generated 283 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:13.364540 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32f348-4ab2-40c9-8276-a31514ff44ba admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:13.380010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32f348-4ab2-40c9-8276-a31514ff44ba admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:13.392730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b526ecf-54fd-42f0-98b6-5f9a481e82e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:13.393505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b526ecf-54fd-42f0-98b6-5f9a481e82e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:13.393792 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b526ecf-54fd-42f0-98b6-5f9a481e82e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:13.396000 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b526ecf-54fd-42f0-98b6-5f9a481e82e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:13.396714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b526ecf-54fd-42f0-98b6-5f9a481e82e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:13.401925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b526ecf-54fd-42f0-98b6-5f9a481e82e4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:13.412724 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:13.430630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 873/1743] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:32:13.435322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff86bf-b9c2-474a-ba7e-dbcb31178cbf admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:13.447735 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff86bf-b9c2-474a-ba7e-dbcb31178cbf admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:13.463845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf0850-8f0a-43d3-89a4-153ab2312841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:13.469216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf0850-8f0a-43d3-89a4-153ab2312841 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:13.470177 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf0850-8f0a-43d3-89a4-153ab2312841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:13.470662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf0850-8f0a-43d3-89a4-153ab2312841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:13.471742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf0850-8f0a-43d3-89a4-153ab2312841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:13.474511 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf0850-8f0a-43d3-89a4-153ab2312841 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:13.477124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:13.507472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 871/1744] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:13.510196 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b526ecf-54fd-42f0-98b6-5f9a481e82e4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:13.524201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b526ecf-54fd-42f0-98b6-5f9a481e82e4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:13.549355 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e76749-5de8-4639-b6a6-6b961d1d6da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:13.550220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e76749-5de8-4639-b6a6-6b961d1d6da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:13.550573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e76749-5de8-4639-b6a6-6b961d1d6da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:13.550853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e76749-5de8-4639-b6a6-6b961d1d6da4 admin admin] PATH_INFO: `/v3/projects/a55fe74586ee4194b631bf092faad938/users/7532c91697d14a0fb9e360af6878c5dd/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:13.594386 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e76749-5de8-4639-b6a6-6b961d1d6da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a55fe74586ee4194b631bf092faad938, user_id=7532c91697d14a0fb9e360af6878c5dd, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:13.598472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e76749-5de8-4639-b6a6-6b961d1d6da4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:13.640238 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 874/1745] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:13 2022] PUT /identity/v3/projects/a55fe74586ee4194b631bf092faad938/users/7532c91697d14a0fb9e360af6878c5dd/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:13.644977 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3b2d9-69a1-4f2c-9b61-5048ec1255fc None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:13.645282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3b2d9-69a1-4f2c-9b61-5048ec1255fc None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:13.645532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3b2d9-69a1-4f2c-9b61-5048ec1255fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:13.684292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-03d3b2d9-69a1-4f2c-9b61-5048ec1255fc None None] MFA Rules not processed for user `7532c91697d14a0fb9e360af6878c5dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:13.685767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-03d3b2d9-69a1-4f2c-9b61-5048ec1255fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:13.837549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 872/1746] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:13.842197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e76749-5de8-4639-b6a6-6b961d1d6da4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:13.943682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e76749-5de8-4639-b6a6-6b961d1d6da4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:13.950036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21661e92-9638-476b-bbca-fde37346ff54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:13.950659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21661e92-9638-476b-bbca-fde37346ff54 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:13.953662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21661e92-9638-476b-bbca-fde37346ff54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:13.954130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21661e92-9638-476b-bbca-fde37346ff54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:13.954926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21661e92-9638-476b-bbca-fde37346ff54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:13.957307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21661e92-9638-476b-bbca-fde37346ff54 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:14.013897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 875/1747] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:13 2022] POST /identity/v3/projects => generated 365 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:14.017470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3b2d9-69a1-4f2c-9b61-5048ec1255fc None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:14.036191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3b2d9-69a1-4f2c-9b61-5048ec1255fc None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:14.180191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7182b7-7689-49c5-babf-9302c514854f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:14.181013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7182b7-7689-49c5-babf-9302c514854f admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:14.181824 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7182b7-7689-49c5-babf-9302c514854f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:14.183482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7182b7-7689-49c5-babf-9302c514854f admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:14.185714 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7182b7-7689-49c5-babf-9302c514854f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:14.190257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7182b7-7689-49c5-babf-9302c514854f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:14.230111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 873/1748] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:14 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:14.233981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21661e92-9638-476b-bbca-fde37346ff54 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:14.250422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21661e92-9638-476b-bbca-fde37346ff54 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:14.263183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc188f3-970b-44d0-89ef-0215beae090a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:14.266371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc188f3-970b-44d0-89ef-0215beae090a admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:14.267497 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc188f3-970b-44d0-89ef-0215beae090a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:14.267790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc188f3-970b-44d0-89ef-0215beae090a admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:14.268478 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc188f3-970b-44d0-89ef-0215beae090a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:14.270960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc188f3-970b-44d0-89ef-0215beae090a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:14.318032 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 876/1749] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:14.322211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7182b7-7689-49c5-babf-9302c514854f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:14.337246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7182b7-7689-49c5-babf-9302c514854f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:14.368253 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b6eb2-1af2-42bf-be87-9b5f901a72ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:14.369106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b6eb2-1af2-42bf-be87-9b5f901a72ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:14.369589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b6eb2-1af2-42bf-be87-9b5f901a72ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:14.370707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b6eb2-1af2-42bf-be87-9b5f901a72ca admin admin] PATH_INFO: `/v3/projects/79a0a018559646969a06d65d2253541b/users/0038202f8d7f47dea5b52783e623f5d6/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:14.426479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b6eb2-1af2-42bf-be87-9b5f901a72ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a0a018559646969a06d65d2253541b, user_id=0038202f8d7f47dea5b52783e623f5d6, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:14.429182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b6eb2-1af2-42bf-be87-9b5f901a72ca admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:14.492367 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 874/1750] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:14 2022] PUT /identity/v3/projects/79a0a018559646969a06d65d2253541b/users/0038202f8d7f47dea5b52783e623f5d6/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:14.497400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc188f3-970b-44d0-89ef-0215beae090a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:14.515891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc188f3-970b-44d0-89ef-0215beae090a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:14.644341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce1c3e-4907-4892-91e0-c1d0e7ff3c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:14.645171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce1c3e-4907-4892-91e0-c1d0e7ff3c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:14.645525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce1c3e-4907-4892-91e0-c1d0e7ff3c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:14.646005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce1c3e-4907-4892-91e0-c1d0e7ff3c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:14.646739 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce1c3e-4907-4892-91e0-c1d0e7ff3c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:14.648455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce1c3e-4907-4892-91e0-c1d0e7ff3c39 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:14.665116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 877/1751] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:14.670013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b6eb2-1af2-42bf-be87-9b5f901a72ca admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:14.681728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b6eb2-1af2-42bf-be87-9b5f901a72ca admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:14.689146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5e472-97ea-4d40-ac0d-122296ad5151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:14.689692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5e472-97ea-4d40-ac0d-122296ad5151 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:14.689978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5e472-97ea-4d40-ac0d-122296ad5151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:14.690254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5e472-97ea-4d40-ac0d-122296ad5151 admin admin] PATH_INFO: `/v3/projects/79a0a018559646969a06d65d2253541b/users/0038202f8d7f47dea5b52783e623f5d6/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:14.692498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5e472-97ea-4d40-ac0d-122296ad5151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a0a018559646969a06d65d2253541b, user_id=0038202f8d7f47dea5b52783e623f5d6, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:14.695280 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5e472-97ea-4d40-ac0d-122296ad5151 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:14.709231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 875/1752] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:14 2022] PUT /identity/v3/projects/79a0a018559646969a06d65d2253541b/users/0038202f8d7f47dea5b52783e623f5d6/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:14.721193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09c33b-f1b4-491e-8dbc-797bb3171a9c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:14.721497 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09c33b-f1b4-491e-8dbc-797bb3171a9c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:14.721758 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09c33b-f1b4-491e-8dbc-797bb3171a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:14.806316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bd09c33b-f1b4-491e-8dbc-797bb3171a9c None None] MFA Rules not processed for user `0038202f8d7f47dea5b52783e623f5d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:14.808190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bd09c33b-f1b4-491e-8dbc-797bb3171a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:14.877234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:14.933166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 878/1753] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:14.949547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5e472-97ea-4d40-ac0d-122296ad5151 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:14.951386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f2f5e472-97ea-4d40-ac0d-122296ad5151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:15.025234 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:15.045628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5e472-97ea-4d40-ac0d-122296ad5151 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:15.049503 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bd982-0ee5-40ae-8425-1e8832a6dd45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:15.050444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bd982-0ee5-40ae-8425-1e8832a6dd45 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:15.050773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bd982-0ee5-40ae-8425-1e8832a6dd45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:15.051065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bd982-0ee5-40ae-8425-1e8832a6dd45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:15.065455 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bd982-0ee5-40ae-8425-1e8832a6dd45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:15.066920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bd982-0ee5-40ae-8425-1e8832a6dd45 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:15.083816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 876/1754] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:15.222266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09c33b-f1b4-491e-8dbc-797bb3171a9c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:15.254852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09c33b-f1b4-491e-8dbc-797bb3171a9c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:15.271341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18981441-1e99-4743-a175-edd1d30f92ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:15.272001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18981441-1e99-4743-a175-edd1d30f92ad service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:15.272288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18981441-1e99-4743-a175-edd1d30f92ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:15.272562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18981441-1e99-4743-a175-edd1d30f92ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:15.428135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18981441-1e99-4743-a175-edd1d30f92ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:15.430035 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18981441-1e99-4743-a175-edd1d30f92ad service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:15.454173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 879/1755] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:16.038884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d05d75-e81d-4aec-9425-15d471e626ab None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:16.039884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d05d75-e81d-4aec-9425-15d471e626ab None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:16.040366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d05d75-e81d-4aec-9425-15d471e626ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:16.185428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d4d05d75-e81d-4aec-9425-15d471e626ab None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:16.187155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d4d05d75-e81d-4aec-9425-15d471e626ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:16.324089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 877/1756] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:32:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:16.526079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18981441-1e99-4743-a175-edd1d30f92ad service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:16.528124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-18981441-1e99-4743-a175-edd1d30f92ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:16.809317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18981441-1e99-4743-a175-edd1d30f92ad service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:16.821134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74f7fa-9a1e-463d-bccf-c64a078e2d94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:16.821962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f7fa-9a1e-463d-bccf-c64a078e2d94 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:16.824945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f7fa-9a1e-463d-bccf-c64a078e2d94 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:16.825388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f7fa-9a1e-463d-bccf-c64a078e2d94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:16.851807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74f7fa-9a1e-463d-bccf-c64a078e2d94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:16.856534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74f7fa-9a1e-463d-bccf-c64a078e2d94 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:16.920706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 880/1757] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:32:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:32:19.430226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d05d75-e81d-4aec-9425-15d471e626ab None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:19.434370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d4d05d75-e81d-4aec-9425-15d471e626ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:19.468152 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74f7fa-9a1e-463d-bccf-c64a078e2d94 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:19.563448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:19.593618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74f7fa-9a1e-463d-bccf-c64a078e2d94 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:19.594014 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d05d75-e81d-4aec-9425-15d471e626ab None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:19.600428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1546ab9-fed5-495d-87d5-b543c00539de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:19.601116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1546ab9-fed5-495d-87d5-b543c00539de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:19.601317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ccc45-fa0b-4589-9a32-6498d6b59374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:19.601430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1546ab9-fed5-495d-87d5-b543c00539de admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:19.601650 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1546ab9-fed5-495d-87d5-b543c00539de admin admin] PATH_INFO: `/v3/users/7306a5753d574f3aa259a5ee2bb8c210` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:19.602378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ccc45-fa0b-4589-9a32-6498d6b59374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:19.602916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ccc45-fa0b-4589-9a32-6498d6b59374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:19.603200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ccc45-fa0b-4589-9a32-6498d6b59374 admin admin] PATH_INFO: `/v3/users/7532c91697d14a0fb9e360af6878c5dd` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:19.603637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1546ab9-fed5-495d-87d5-b543c00539de admin admin] RBAC: Authorizing `identity:delete_user(user_id=7306a5753d574f3aa259a5ee2bb8c210)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:19.605871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ccc45-fa0b-4589-9a32-6498d6b59374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7532c91697d14a0fb9e360af6878c5dd)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:19.606303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1546ab9-fed5-495d-87d5-b543c00539de admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:19.610988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ccc45-fa0b-4589-9a32-6498d6b59374 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:19.689889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-093ccc45-fa0b-4589-9a32-6498d6b59374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7532c91697d14a0fb9e360af6878c5dd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:19.691462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-093ccc45-fa0b-4589-9a32-6498d6b59374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7532c91697d14a0fb9e360af6878c5dd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:19.692563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-093ccc45-fa0b-4589-9a32-6498d6b59374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7532c91697d14a0fb9e360af6878c5dd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:19.693298 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c1546ab9-fed5-495d-87d5-b543c00539de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7306a5753d574f3aa259a5ee2bb8c210'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:19.693968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c1546ab9-fed5-495d-87d5-b543c00539de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7306a5753d574f3aa259a5ee2bb8c210'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:19.694748 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c1546ab9-fed5-495d-87d5-b543c00539de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7306a5753d574f3aa259a5ee2bb8c210'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:19.702417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:19.704516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c1546ab9-fed5-495d-87d5-b543c00539de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7306a5753d574f3aa259a5ee2bb8c210'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:19.715449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-093ccc45-fa0b-4589-9a32-6498d6b59374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7532c91697d14a0fb9e360af6878c5dd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:19.724127 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-093ccc45-fa0b-4589-9a32-6498d6b59374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7532c91697d14a0fb9e360af6878c5dd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:19.733085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c1546ab9-fed5-495d-87d5-b543c00539de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7306a5753d574f3aa259a5ee2bb8c210'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:19.745690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 881/1758] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:19 2022] DELETE /identity/v3/users/7532c91697d14a0fb9e360af6878c5dd => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:19.748003 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff792f-2e4f-403a-942a-368780d68a54 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:19.749091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff792f-2e4f-403a-942a-368780d68a54 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:19.749372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff792f-2e4f-403a-942a-368780d68a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:19.771970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 878/1759] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:19 2022] DELETE /identity/v3/users/7306a5753d574f3aa259a5ee2bb8c210 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:19.782659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd16a9-8ebf-4ee5-a368-a5edf7a2254e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:19.783014 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd16a9-8ebf-4ee5-a368-a5edf7a2254e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:19.783303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd16a9-8ebf-4ee5-a368-a5edf7a2254e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:19.787233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bcff792f-2e4f-403a-942a-368780d68a54 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:19.789422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bcff792f-2e4f-403a-942a-368780d68a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:19.818718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-2cbd16a9-8ebf-4ee5-a368-a5edf7a2254e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:19.820147 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2cbd16a9-8ebf-4ee5-a368-a5edf7a2254e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:19.879324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 882/1760] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:19.891316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff792f-2e4f-403a-942a-368780d68a54 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:19.893521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bcff792f-2e4f-403a-942a-368780d68a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:19.931063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 879/1761] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:19.953998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd16a9-8ebf-4ee5-a368-a5edf7a2254e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:19.955551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2cbd16a9-8ebf-4ee5-a368-a5edf7a2254e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:19.990358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff792f-2e4f-403a-942a-368780d68a54 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:19.998408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3c3cb-9aff-4ecb-8338-2dd64ccd64b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:19.999060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3c3cb-9aff-4ecb-8338-2dd64ccd64b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:19.999386 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3c3cb-9aff-4ecb-8338-2dd64ccd64b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:19.999607 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3c3cb-9aff-4ecb-8338-2dd64ccd64b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:20.019071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3c3cb-9aff-4ecb-8338-2dd64ccd64b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:20.020541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3c3cb-9aff-4ecb-8338-2dd64ccd64b0 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:20.031897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 883/1762] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:20.052360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd16a9-8ebf-4ee5-a368-a5edf7a2254e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:20.058282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c537dd-ade1-4f7f-9029-0000c2adbfad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:20.059027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c537dd-ade1-4f7f-9029-0000c2adbfad service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:20.059530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c537dd-ade1-4f7f-9029-0000c2adbfad service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:20.059855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c537dd-ade1-4f7f-9029-0000c2adbfad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:20.073856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c537dd-ade1-4f7f-9029-0000c2adbfad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:20.081850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c537dd-ade1-4f7f-9029-0000c2adbfad service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:20.099576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 880/1763] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:20.355913 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3c3cb-9aff-4ecb-8338-2dd64ccd64b0 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:20.357643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b2b3c3cb-9aff-4ecb-8338-2dd64ccd64b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:20.369191 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:20.375482 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3c3cb-9aff-4ecb-8338-2dd64ccd64b0 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:20.382951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:20.383674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:20.384711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:20.384711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] PATH_INFO: `/v3/users/f8d400134bed49288b8cda35ce1ebe40` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:20.386254 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d400134bed49288b8cda35ce1ebe40)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:20.393900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:20.483077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d400134bed49288b8cda35ce1ebe40'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:20.483706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d400134bed49288b8cda35ce1ebe40'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:20.485453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d400134bed49288b8cda35ce1ebe40'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:20.542534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d400134bed49288b8cda35ce1ebe40'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:20.564019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d400134bed49288b8cda35ce1ebe40'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:20.607420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 884/1764] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:20 2022] DELETE /identity/v3/users/f8d400134bed49288b8cda35ce1ebe40 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:20.611952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19601dea-3d97-4fab-896e-2d4c6db37622 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:20.612567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19601dea-3d97-4fab-896e-2d4c6db37622 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:20.612567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19601dea-3d97-4fab-896e-2d4c6db37622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:20.667818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-19601dea-3d97-4fab-896e-2d4c6db37622 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:20.669104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-19601dea-3d97-4fab-896e-2d4c6db37622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:20.780719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:20.782363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:20.827846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 881/1765] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:20.830129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19601dea-3d97-4fab-896e-2d4c6db37622 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:20.836484 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-19601dea-3d97-4fab-896e-2d4c6db37622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:20.861830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19601dea-3d97-4fab-896e-2d4c6db37622 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:20.877299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:20.877299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:20.877299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:20.877299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] PATH_INFO: `/v3/projects/a69355df1a0d4be69b77d7688a01e325` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:20.882137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a69355df1a0d4be69b77d7688a01e325)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:20.882494 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:20.910574 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:20.922033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f72b0-576c-4865-abf7-6675ac51c3b0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:20.927909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee1ed2-6b39-4214-9306-47cdbaa27f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:20.928534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee1ed2-6b39-4214-9306-47cdbaa27f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:20.928949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee1ed2-6b39-4214-9306-47cdbaa27f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:20.929251 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee1ed2-6b39-4214-9306-47cdbaa27f39 admin admin] PATH_INFO: `/v3/projects/a55fe74586ee4194b631bf092faad938` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:20.933849 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee1ed2-6b39-4214-9306-47cdbaa27f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a55fe74586ee4194b631bf092faad938)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:20.936156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee1ed2-6b39-4214-9306-47cdbaa27f39 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:21.011331 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a69355df1a0d4be69b77d7688a01e325', 'request_id': 'req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.012168 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-afee1ed2-6b39-4214-9306-47cdbaa27f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a55fe74586ee4194b631bf092faad938', 'request_id': 'req-afee1ed2-6b39-4214-9306-47cdbaa27f39'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.018333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a69355df1a0d4be69b77d7688a01e325', 'request_id': 'req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.020056 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-afee1ed2-6b39-4214-9306-47cdbaa27f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a55fe74586ee4194b631bf092faad938', 'request_id': 'req-afee1ed2-6b39-4214-9306-47cdbaa27f39'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.025168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a69355df1a0d4be69b77d7688a01e325', 'request_id': 'req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.031663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-afee1ed2-6b39-4214-9306-47cdbaa27f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a55fe74586ee4194b631bf092faad938', 'request_id': 'req-afee1ed2-6b39-4214-9306-47cdbaa27f39'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.044019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] The token cache is being invalidate because project a69355df1a0d4be69b77d7688a01e325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:21.044810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.045631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.049171 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 882/1766] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:20 2022] DELETE /identity/v3/projects/a69355df1a0d4be69b77d7688a01e325 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:21.049784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:21.053462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:21.073608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-afee1ed2-6b39-4214-9306-47cdbaa27f39 admin admin] The token cache is being invalidate because project a55fe74586ee4194b631bf092faad938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:21.073608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-afee1ed2-6b39-4214-9306-47cdbaa27f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.073608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-afee1ed2-6b39-4214-9306-47cdbaa27f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.073608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 885/1767] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:20 2022] DELETE /identity/v3/projects/a55fe74586ee4194b631bf092faad938 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:21.073608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e02d5-0edd-4bea-9edc-6e5d670e4d98 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:21.074045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e02d5-0edd-4bea-9edc-6e5d670e4d98 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:21.074045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e02d5-0edd-4bea-9edc-6e5d670e4d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:21.083147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:21.099059 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b74e02d5-0edd-4bea-9edc-6e5d670e4d98 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:21.100413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b74e02d5-0edd-4bea-9edc-6e5d670e4d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:21.148953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17309ff4-ad38-46bc-a977-d4fd96c3d9c2 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:21.174063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 886/1768] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:21.175234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e02d5-0edd-4bea-9edc-6e5d670e4d98 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:21.177257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b74e02d5-0edd-4bea-9edc-6e5d670e4d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:21.192469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e02d5-0edd-4bea-9edc-6e5d670e4d98 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:21.198448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:21.199098 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:21.199378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:21.199648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] PATH_INFO: `/v3/users/0038202f8d7f47dea5b52783e623f5d6` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:21.200884 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0038202f8d7f47dea5b52783e623f5d6)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:21.203342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:21.212734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1893a30-b399-4bea-a3ca-ab65e1cc0f9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:21.214207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1893a30-b399-4bea-a3ca-ab65e1cc0f9f service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:21.214773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1893a30-b399-4bea-a3ca-ab65e1cc0f9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:21.215117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1893a30-b399-4bea-a3ca-ab65e1cc0f9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:21.217129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e1893a30-b399-4bea-a3ca-ab65e1cc0f9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:21.245123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1893a30-b399-4bea-a3ca-ab65e1cc0f9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:21.247612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1893a30-b399-4bea-a3ca-ab65e1cc0f9f service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:21.262998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0038202f8d7f47dea5b52783e623f5d6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.263682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0038202f8d7f47dea5b52783e623f5d6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.264370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 883/1769] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:21.264495 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0038202f8d7f47dea5b52783e623f5d6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.266122 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd1f14-0584-4605-a7c3-ddd0fdbf5a1e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:21.266410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd1f14-0584-4605-a7c3-ddd0fdbf5a1e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:21.267149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd1f14-0584-4605-a7c3-ddd0fdbf5a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:21.280324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0038202f8d7f47dea5b52783e623f5d6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.288394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0038202f8d7f47dea5b52783e623f5d6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:21.296500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-babd1f14-0584-4605-a7c3-ddd0fdbf5a1e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:21.297912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-babd1f14-0584-4605-a7c3-ddd0fdbf5a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:21.306768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 884/1770] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:21.309642 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 887/1771] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:21 2022] DELETE /identity/v3/users/0038202f8d7f47dea5b52783e623f5d6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:21.314651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd1f14-0584-4605-a7c3-ddd0fdbf5a1e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:21.338835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd1f14-0584-4605-a7c3-ddd0fdbf5a1e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:21.437243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:21.452734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d34555-b103-4fa7-aa03-c9d10fa625f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:21.455246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d34555-b103-4fa7-aa03-c9d10fa625f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:21.455529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d34555-b103-4fa7-aa03-c9d10fa625f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:21.455887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d34555-b103-4fa7-aa03-c9d10fa625f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:21.456578 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d34555-b103-4fa7-aa03-c9d10fa625f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:21.458219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d34555-b103-4fa7-aa03-c9d10fa625f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:21.462764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d34555-b103-4fa7-aa03-c9d10fa625f6 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:21.480885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 885/1772] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:32:21.485613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:21.500437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6cf1e-f813-4600-84fc-8fbc4db85f34 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:21.511984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b1912-6869-4dc2-8e0c-ee34e7bd4260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:21.512776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b1912-6869-4dc2-8e0c-ee34e7bd4260 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:21.513120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b1912-6869-4dc2-8e0c-ee34e7bd4260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:21.513462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b1912-6869-4dc2-8e0c-ee34e7bd4260 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:21.514182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b1912-6869-4dc2-8e0c-ee34e7bd4260 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:21.516356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b1912-6869-4dc2-8e0c-ee34e7bd4260 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:21.579610 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 888/1773] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:21 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:21.585649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d34555-b103-4fa7-aa03-c9d10fa625f6 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:21.599201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d34555-b103-4fa7-aa03-c9d10fa625f6 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:21.743586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dea363-7c6c-4a11-8db9-62f3951ee846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:21.744333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dea363-7c6c-4a11-8db9-62f3951ee846 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:21.745499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dea363-7c6c-4a11-8db9-62f3951ee846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:21.745835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dea363-7c6c-4a11-8db9-62f3951ee846 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:21.746637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dea363-7c6c-4a11-8db9-62f3951ee846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:21.749063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dea363-7c6c-4a11-8db9-62f3951ee846 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:21.785471 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 886/1774] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:21 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:21.788695 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b1912-6869-4dc2-8e0c-ee34e7bd4260 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:21.811926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b1912-6869-4dc2-8e0c-ee34e7bd4260 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:21.819588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cfdd8-a702-48cc-9468-e3c514fc4dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:21.820222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cfdd8-a702-48cc-9468-e3c514fc4dcd admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:21.820505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cfdd8-a702-48cc-9468-e3c514fc4dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:21.820837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cfdd8-a702-48cc-9468-e3c514fc4dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:21.832890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6cfdd8-a702-48cc-9468-e3c514fc4dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:21.839601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6cfdd8-a702-48cc-9468-e3c514fc4dcd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:21.864059 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 889/1775] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:21.868844 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dea363-7c6c-4a11-8db9-62f3951ee846 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:21.885424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dea363-7c6c-4a11-8db9-62f3951ee846 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:21.897282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b380945-1c9d-4be5-bcd7-18d6b16f1206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:21.897282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b380945-1c9d-4be5-bcd7-18d6b16f1206 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:21.897282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b380945-1c9d-4be5-bcd7-18d6b16f1206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:21.897282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b380945-1c9d-4be5-bcd7-18d6b16f1206 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:21.898224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b380945-1c9d-4be5-bcd7-18d6b16f1206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:21.900674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b380945-1c9d-4be5-bcd7-18d6b16f1206 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:21.929907 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cfdd8-a702-48cc-9468-e3c514fc4dcd admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:21.930412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 887/1776] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:21.946642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b380945-1c9d-4be5-bcd7-18d6b16f1206 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:21.950898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-1b380945-1c9d-4be5-bcd7-18d6b16f1206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:21.959429 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cfdd8-a702-48cc-9468-e3c514fc4dcd admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:21.977835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b380945-1c9d-4be5-bcd7-18d6b16f1206 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:21.988165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:21.988397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:21.990814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:21.991133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9 admin admin] PATH_INFO: `/v3/projects/fcd3d475eebf4f21988e04d3b6e2e0e3` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:21.993611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd3d475eebf4f21988e04d3b6e2e0e3)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:22.000560 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:22.023996 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66045588-27c8-4a2b-b371-3e0b5efd16b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:22.027952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66045588-27c8-4a2b-b371-3e0b5efd16b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:22.028243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66045588-27c8-4a2b-b371-3e0b5efd16b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:22.033559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66045588-27c8-4a2b-b371-3e0b5efd16b4 admin admin] PATH_INFO: `/v3/projects/9440563c215d40a7b7371c784c2449af/users/619e3cb8198c460db8bf8292fb123597/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:22.114812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66045588-27c8-4a2b-b371-3e0b5efd16b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9440563c215d40a7b7371c784c2449af, user_id=619e3cb8198c460db8bf8292fb123597, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:22.124259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd3d475eebf4f21988e04d3b6e2e0e3', 'request_id': 'req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.127764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66045588-27c8-4a2b-b371-3e0b5efd16b4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:22.137125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd3d475eebf4f21988e04d3b6e2e0e3', 'request_id': 'req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.145523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd3d475eebf4f21988e04d3b6e2e0e3', 'request_id': 'req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.165103 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 890/1777] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:21 2022] PUT /identity/v3/projects/9440563c215d40a7b7371c784c2449af/users/619e3cb8198c460db8bf8292fb123597/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:22.166910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66045588-27c8-4a2b-b371-3e0b5efd16b4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:22.173427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-66045588-27c8-4a2b-b371-3e0b5efd16b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:22.183276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9 admin admin] The token cache is being invalidate because project fcd3d475eebf4f21988e04d3b6e2e0e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:22.183928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.187863 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-74cd9a78-9004-4f8e-94d5-2c3f9cda7bd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.191120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 888/1778] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:21 2022] DELETE /identity/v3/projects/fcd3d475eebf4f21988e04d3b6e2e0e3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:22.192435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77163f4-82bb-4f7d-b209-e101f122d682 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:22.192729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77163f4-82bb-4f7d-b209-e101f122d682 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:22.193034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77163f4-82bb-4f7d-b209-e101f122d682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:22.223477 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:22.253746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d77163f4-82bb-4f7d-b209-e101f122d682 None None] MFA Rules not processed for user `619e3cb8198c460db8bf8292fb123597`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:22.255010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d77163f4-82bb-4f7d-b209-e101f122d682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:22.281627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66045588-27c8-4a2b-b371-3e0b5efd16b4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:22.293755 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:22.294711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:22.295049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:22.295330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] PATH_INFO: `/v3/projects/79a0a018559646969a06d65d2253541b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:22.296480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79a0a018559646969a06d65d2253541b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:22.300896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:22.305200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:22.388891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79a0a018559646969a06d65d2253541b', 'request_id': 'req-c02aba38-f2fa-4516-bcf4-56c81df8f016'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.389696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 889/1779] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:22.395093 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77163f4-82bb-4f7d-b209-e101f122d682 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:22.399239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d77163f4-82bb-4f7d-b209-e101f122d682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:22.400001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79a0a018559646969a06d65d2253541b', 'request_id': 'req-c02aba38-f2fa-4516-bcf4-56c81df8f016'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.409767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79a0a018559646969a06d65d2253541b', 'request_id': 'req-c02aba38-f2fa-4516-bcf4-56c81df8f016'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.423958 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77163f4-82bb-4f7d-b209-e101f122d682 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:22.446220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] The token cache is being invalidate because project 79a0a018559646969a06d65d2253541b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:22.446616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.447507 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.453051 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 891/1780] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:22 2022] DELETE /identity/v3/projects/79a0a018559646969a06d65d2253541b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:22.453952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:22.458210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:22.514673 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:22.515836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:22.516319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:22.516802 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] PATH_INFO: `/v3/users/f93f88ceddc940c38170e9067d431e4f` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:22.518664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f93f88ceddc940c38170e9067d431e4f)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:22.520865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:22.594366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02aba38-f2fa-4516-bcf4-56c81df8f016 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:22.608107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f93f88ceddc940c38170e9067d431e4f'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.609289 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f93f88ceddc940c38170e9067d431e4f'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.610287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f93f88ceddc940c38170e9067d431e4f'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.610416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce6d2b-546d-432c-95cc-c83b105e66d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:22.611082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce6d2b-546d-432c-95cc-c83b105e66d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:22.611432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce6d2b-546d-432c-95cc-c83b105e66d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:22.611827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce6d2b-546d-432c-95cc-c83b105e66d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:22.612481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce6d2b-546d-432c-95cc-c83b105e66d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:22.614964 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce6d2b-546d-432c-95cc-c83b105e66d8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:22.622342 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f93f88ceddc940c38170e9067d431e4f'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.642359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f93f88ceddc940c38170e9067d431e4f'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:22.664814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 892/1781] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:22 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:22.664814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df894042-1249-48b8-bdbf-024d2f687fc7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:22.664814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df894042-1249-48b8-bdbf-024d2f687fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:22.664814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df894042-1249-48b8-bdbf-024d2f687fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:22.668797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 890/1782] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:22 2022] DELETE /identity/v3/users/f93f88ceddc940c38170e9067d431e4f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:22.668797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:22.689091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-df894042-1249-48b8-bdbf-024d2f687fc7 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:22.690566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-df894042-1249-48b8-bdbf-024d2f687fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:22.692009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:22.746218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:22.813957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:22.815488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 893/1783] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:22.823003 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607bf5f-e487-4430-9bf0-57233fbbeceb None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:22.823502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607bf5f-e487-4430-9bf0-57233fbbeceb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:22.824337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607bf5f-e487-4430-9bf0-57233fbbeceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:22.833103 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a8c25-c942-4824-a0b4-bd48d2c5f484 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:22.843459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7564e-95bc-4e7a-9aed-66974fd528ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:22.845906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7564e-95bc-4e7a-9aed-66974fd528ea admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:22.846108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7564e-95bc-4e7a-9aed-66974fd528ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:22.846390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7564e-95bc-4e7a-9aed-66974fd528ea admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:22.847055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7564e-95bc-4e7a-9aed-66974fd528ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:22.853462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7564e-95bc-4e7a-9aed-66974fd528ea admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:22.862873 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c607bf5f-e487-4430-9bf0-57233fbbeceb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:22.864289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c607bf5f-e487-4430-9bf0-57233fbbeceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:22.879045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 894/1784] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:22.890394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607bf5f-e487-4430-9bf0-57233fbbeceb None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:22.903583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 891/1785] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:22 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:22.907979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7564e-95bc-4e7a-9aed-66974fd528ea admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:22.909657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607bf5f-e487-4430-9bf0-57233fbbeceb None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:22.915291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9a21a-41e9-4bfe-b45c-f7bc81921a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:22.916046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9a21a-41e9-4bfe-b45c-f7bc81921a3f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:22.916384 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9a21a-41e9-4bfe-b45c-f7bc81921a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:22.917411 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9a21a-41e9-4bfe-b45c-f7bc81921a3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:22.918151 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9a21a-41e9-4bfe-b45c-f7bc81921a3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:22.918527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9a21a-41e9-4bfe-b45c-f7bc81921a3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:22.920236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9a21a-41e9-4bfe-b45c-f7bc81921a3f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:22.931705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7564e-95bc-4e7a-9aed-66974fd528ea admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:22.939810 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 895/1786] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:32:22.944291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9a21a-41e9-4bfe-b45c-f7bc81921a3f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:22.965782 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690994b-bab0-4f69-b0f3-8d1c02e7e70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:22.966419 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690994b-bab0-4f69-b0f3-8d1c02e7e70c admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:22.966700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690994b-bab0-4f69-b0f3-8d1c02e7e70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:22.966974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690994b-bab0-4f69-b0f3-8d1c02e7e70c admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:22.967628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690994b-bab0-4f69-b0f3-8d1c02e7e70c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:22.968016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9a21a-41e9-4bfe-b45c-f7bc81921a3f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:22.969274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690994b-bab0-4f69-b0f3-8d1c02e7e70c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:22.973927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635260f-2892-4a9a-985a-8854e6f33923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:22.974564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635260f-2892-4a9a-985a-8854e6f33923 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:22.974848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635260f-2892-4a9a-985a-8854e6f33923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:22.975173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635260f-2892-4a9a-985a-8854e6f33923 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:22.975819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635260f-2892-4a9a-985a-8854e6f33923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:22.978137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635260f-2892-4a9a-985a-8854e6f33923 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:23.005296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 892/1787] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:32:23.010922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690994b-bab0-4f69-b0f3-8d1c02e7e70c admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:23.033122 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 896/1788] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:22 2022] POST /identity/v3/projects => generated 387 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:23.034724 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635260f-2892-4a9a-985a-8854e6f33923 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:23.036475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690994b-bab0-4f69-b0f3-8d1c02e7e70c admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:23.123778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635260f-2892-4a9a-985a-8854e6f33923 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:23.138092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163ad92-36b9-4b8e-9a7c-8130027bb962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:23.138092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163ad92-36b9-4b8e-9a7c-8130027bb962 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:23.138092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163ad92-36b9-4b8e-9a7c-8130027bb962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:23.140878 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163ad92-36b9-4b8e-9a7c-8130027bb962 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:23.142073 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163ad92-36b9-4b8e-9a7c-8130027bb962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:23.145686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163ad92-36b9-4b8e-9a7c-8130027bb962 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:23.165451 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b8794-3468-4cdc-990e-1d452539c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:23.166373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b8794-3468-4cdc-990e-1d452539c225 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:23.167608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b8794-3468-4cdc-990e-1d452539c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:23.168091 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b8794-3468-4cdc-990e-1d452539c225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:23.173363 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b8794-3468-4cdc-990e-1d452539c225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:23.175049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b8794-3468-4cdc-990e-1d452539c225 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:23.201575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 893/1789] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:23.205736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b8794-3468-4cdc-990e-1d452539c225 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:23.211289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 897/1790] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:23 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:23.218270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163ad92-36b9-4b8e-9a7c-8130027bb962 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:23.231864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b8794-3468-4cdc-990e-1d452539c225 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:23.243832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebac3f7-2cf7-41a3-a607-7f92539514ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:23.243832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebac3f7-2cf7-41a3-a607-7f92539514ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:23.243832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebac3f7-2cf7-41a3-a607-7f92539514ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:23.243832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebac3f7-2cf7-41a3-a607-7f92539514ff admin admin] PATH_INFO: `/v3/projects/6654a795f2c747e68bd6f599e0819071/users/9cc723d89c1d4b1e92e619fca0b9c1cf/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:23.255908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163ad92-36b9-4b8e-9a7c-8130027bb962 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:23.269891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7764ff5-fa51-4cc7-95d1-1da94739405d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:23.270862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7764ff5-fa51-4cc7-95d1-1da94739405d admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:23.271633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7764ff5-fa51-4cc7-95d1-1da94739405d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:23.271633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7764ff5-fa51-4cc7-95d1-1da94739405d admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:23.272633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7764ff5-fa51-4cc7-95d1-1da94739405d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:23.274447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7764ff5-fa51-4cc7-95d1-1da94739405d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:23.286481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebac3f7-2cf7-41a3-a607-7f92539514ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=6654a795f2c747e68bd6f599e0819071, user_id=9cc723d89c1d4b1e92e619fca0b9c1cf, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:23.290396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebac3f7-2cf7-41a3-a607-7f92539514ff admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:23.305910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 898/1791] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:23.307311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7764ff5-fa51-4cc7-95d1-1da94739405d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:23.319198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7764ff5-fa51-4cc7-95d1-1da94739405d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:23.332343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 894/1792] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:23 2022] PUT /identity/v3/projects/6654a795f2c747e68bd6f599e0819071/users/9cc723d89c1d4b1e92e619fca0b9c1cf/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:23.333523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebac3f7-2cf7-41a3-a607-7f92539514ff admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:23.344002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6ebac3f7-2cf7-41a3-a607-7f92539514ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:23.459769 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3f240-a20a-4cc7-b683-cef49d1abf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:23.460979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3f240-a20a-4cc7-b683-cef49d1abf78 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:23.461499 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3f240-a20a-4cc7-b683-cef49d1abf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:23.461984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3f240-a20a-4cc7-b683-cef49d1abf78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:23.462636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3f240-a20a-4cc7-b683-cef49d1abf78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:23.466219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3f240-a20a-4cc7-b683-cef49d1abf78 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:23.488235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 899/1793] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:23.495564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba00dbe-afc5-45fd-9783-cef344800e15 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:23.496287 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba00dbe-afc5-45fd-9783-cef344800e15 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:23.496712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba00dbe-afc5-45fd-9783-cef344800e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:23.518066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebac3f7-2cf7-41a3-a607-7f92539514ff admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:23.553476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:23.554029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:23.554807 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:23.555164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:23.555845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] PATH_INFO: `/v3/projects/baf4dfe69cb0461fa060c2662766648b` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:23.560032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=baf4dfe69cb0461fa060c2662766648b)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:23.562467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:23.569294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-eba00dbe-afc5-45fd-9783-cef344800e15 None None] MFA Rules not processed for user `9cc723d89c1d4b1e92e619fca0b9c1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:23.570666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-eba00dbe-afc5-45fd-9783-cef344800e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:23.665656 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:23.667921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baf4dfe69cb0461fa060c2662766648b', 'request_id': 'req-be36d2db-9192-4855-b597-92ac9865c0ae'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:23.668878 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 900/1794] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:23.670274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba00dbe-afc5-45fd-9783-cef344800e15 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:23.677729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baf4dfe69cb0461fa060c2662766648b', 'request_id': 'req-be36d2db-9192-4855-b597-92ac9865c0ae'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:23.686759 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baf4dfe69cb0461fa060c2662766648b', 'request_id': 'req-be36d2db-9192-4855-b597-92ac9865c0ae'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:23.697505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba00dbe-afc5-45fd-9783-cef344800e15 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:23.708076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] The token cache is being invalidate because project baf4dfe69cb0461fa060c2662766648b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:23.708930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:23.710090 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:23.716844 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 895/1795] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:23 2022] DELETE /identity/v3/projects/baf4dfe69cb0461fa060c2662766648b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:23.718062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:23.719705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:23.806964 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcaba1-264d-4008-9ad7-6bd812a4f0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:23.808969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcaba1-264d-4008-9ad7-6bd812a4f0c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:23.809629 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcaba1-264d-4008-9ad7-6bd812a4f0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:23.810046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcaba1-264d-4008-9ad7-6bd812a4f0c8 admin admin] PATH_INFO: `/v3/projects/3960897483904b508688ce79adfde4ca/users/2984bb939db0458cb9040c57d2f1bbf3/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:23.813510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36d2db-9192-4855-b597-92ac9865c0ae admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:23.819368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c35b2-ecf8-47a6-b68f-94e9b612d670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:23.820324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c35b2-ecf8-47a6-b68f-94e9b612d670 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:23.820674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c35b2-ecf8-47a6-b68f-94e9b612d670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:23.821071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c35b2-ecf8-47a6-b68f-94e9b612d670 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:23.821782 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c35b2-ecf8-47a6-b68f-94e9b612d670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:23.824406 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c35b2-ecf8-47a6-b68f-94e9b612d670 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:23.837125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcaba1-264d-4008-9ad7-6bd812a4f0c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3960897483904b508688ce79adfde4ca, user_id=2984bb939db0458cb9040c57d2f1bbf3, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:23.842360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcaba1-264d-4008-9ad7-6bd812a4f0c8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:23.860495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 896/1796] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:23 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:23.862008 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee8a22-e5a9-4def-b07a-d0a73154c1c8 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:23.862428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee8a22-e5a9-4def-b07a-d0a73154c1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:23.862792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee8a22-e5a9-4def-b07a-d0a73154c1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:23.884427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 901/1797] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:23 2022] PUT /identity/v3/projects/3960897483904b508688ce79adfde4ca/users/2984bb939db0458cb9040c57d2f1bbf3/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:23.884427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcaba1-264d-4008-9ad7-6bd812a4f0c8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:23.892485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b1ee8a22-e5a9-4def-b07a-d0a73154c1c8 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:23.894147 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b1ee8a22-e5a9-4def-b07a-d0a73154c1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:23.959954 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcaba1-264d-4008-9ad7-6bd812a4f0c8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:23.978444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 897/1798] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:23.980050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10b139-8e80-4d4a-8a78-59eb12cea03a None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:23.980329 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10b139-8e80-4d4a-8a78-59eb12cea03a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:23.980646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10b139-8e80-4d4a-8a78-59eb12cea03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:23.991155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:24.013237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-9e10b139-8e80-4d4a-8a78-59eb12cea03a None None] MFA Rules not processed for user `2984bb939db0458cb9040c57d2f1bbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:24.014468 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9e10b139-8e80-4d4a-8a78-59eb12cea03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:24.028956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d255c-41ba-44e9-b8d9-bc1d7e5bed44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:24.029579 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d255c-41ba-44e9-b8d9-bc1d7e5bed44 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:24.029858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d255c-41ba-44e9-b8d9-bc1d7e5bed44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:24.030133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d255c-41ba-44e9-b8d9-bc1d7e5bed44 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:24.030755 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d255c-41ba-44e9-b8d9-bc1d7e5bed44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:24.032686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d255c-41ba-44e9-b8d9-bc1d7e5bed44 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:24.072199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 902/1799] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:23 2022] POST /identity/v3/users => generated 299 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:24.072199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dd195-4cf7-4272-b48e-d01cb9ed401c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:24.072199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dd195-4cf7-4272-b48e-d01cb9ed401c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:24.072199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dd195-4cf7-4272-b48e-d01cb9ed401c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:24.085924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:24.119345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 898/1800] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:24.119345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10b139-8e80-4d4a-8a78-59eb12cea03a None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:24.125122 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c14dd195-4cf7-4272-b48e-d01cb9ed401c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:24.126108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c14dd195-4cf7-4272-b48e-d01cb9ed401c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:24.137984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 903/1801] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:24.139167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dd195-4cf7-4272-b48e-d01cb9ed401c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:24.142029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c14dd195-4cf7-4272-b48e-d01cb9ed401c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:24.149902 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10b139-8e80-4d4a-8a78-59eb12cea03a None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:24.159807 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d764e2a-8e52-408a-bbcf-06124831beaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:24.162521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d764e2a-8e52-408a-bbcf-06124831beaa admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:24.162521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d764e2a-8e52-408a-bbcf-06124831beaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:24.162832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d764e2a-8e52-408a-bbcf-06124831beaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:24.163521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d764e2a-8e52-408a-bbcf-06124831beaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:24.168705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d764e2a-8e52-408a-bbcf-06124831beaa admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:24.176689 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dd195-4cf7-4272-b48e-d01cb9ed401c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:24.185476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481cb78-c133-4455-a800-99595ade9635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:24.186349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481cb78-c133-4455-a800-99595ade9635 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:24.186721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481cb78-c133-4455-a800-99595ade9635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:24.187282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481cb78-c133-4455-a800-99595ade9635 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:24.188085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481cb78-c133-4455-a800-99595ade9635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:24.190326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481cb78-c133-4455-a800-99595ade9635 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:24.197121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 899/1802] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:24.197121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d764e2a-8e52-408a-bbcf-06124831beaa admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:24.209860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d764e2a-8e52-408a-bbcf-06124831beaa admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:24.219034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e2720-2390-4953-b5cd-e67e9a9355de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:24.220009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e2720-2390-4953-b5cd-e67e9a9355de admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:24.220384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e2720-2390-4953-b5cd-e67e9a9355de admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:24.220767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e2720-2390-4953-b5cd-e67e9a9355de admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:24.221804 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e2720-2390-4953-b5cd-e67e9a9355de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:24.222084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e2720-2390-4953-b5cd-e67e9a9355de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:24.241204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e2720-2390-4953-b5cd-e67e9a9355de admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:24.252297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 904/1803] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:24 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:24.253172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481cb78-c133-4455-a800-99595ade9635 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:24.270955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 900/1804] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:32:24.270955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e2720-2390-4953-b5cd-e67e9a9355de admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:24.292630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e2720-2390-4953-b5cd-e67e9a9355de admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:24.461438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde9911-6949-4930-b5e7-091389d1502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:24.462307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde9911-6949-4930-b5e7-091389d1502a admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:24.462582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde9911-6949-4930-b5e7-091389d1502a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:24.462938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde9911-6949-4930-b5e7-091389d1502a admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:24.465738 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde9911-6949-4930-b5e7-091389d1502a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:24.470499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde9911-6949-4930-b5e7-091389d1502a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:24.486063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481cb78-c133-4455-a800-99595ade9635 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:24.503145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45434f0a-4992-4817-99e7-8013e9bad8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:24.504217 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45434f0a-4992-4817-99e7-8013e9bad8e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:24.504796 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45434f0a-4992-4817-99e7-8013e9bad8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:24.505590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45434f0a-4992-4817-99e7-8013e9bad8e4 admin admin] PATH_INFO: `/v3/projects/1c3f443988c1407b8032a8a73e6a16a7/users/008d17d7079e43b7887cfc8792a3f4a2/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:24.536195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 901/1805] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:24 2022] POST /identity/v3/users => generated 300 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:24.537321 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde9911-6949-4930-b5e7-091389d1502a admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:24.539900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45434f0a-4992-4817-99e7-8013e9bad8e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3f443988c1407b8032a8a73e6a16a7, user_id=008d17d7079e43b7887cfc8792a3f4a2, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:24.545195 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45434f0a-4992-4817-99e7-8013e9bad8e4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:24.562280 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde9911-6949-4930-b5e7-091389d1502a admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:24.569095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9eff25-6c02-4e73-bcf7-af25269ee362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:24.569792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9eff25-6c02-4e73-bcf7-af25269ee362 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:24.570328 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9eff25-6c02-4e73-bcf7-af25269ee362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:24.570610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9eff25-6c02-4e73-bcf7-af25269ee362 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:24.571521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9eff25-6c02-4e73-bcf7-af25269ee362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:24.575278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9eff25-6c02-4e73-bcf7-af25269ee362 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:24.627314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 902/1806] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:24 2022] POST /identity/v3/projects => generated 369 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:24.628124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9eff25-6c02-4e73-bcf7-af25269ee362 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:24.632554 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 905/1807] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:24 2022] PUT /identity/v3/projects/1c3f443988c1407b8032a8a73e6a16a7/users/008d17d7079e43b7887cfc8792a3f4a2/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:24.637290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45434f0a-4992-4817-99e7-8013e9bad8e4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:24.782454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9eff25-6c02-4e73-bcf7-af25269ee362 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:24.790404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9075284-e352-4853-8aa7-c54e228402a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:24.792356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9075284-e352-4853-8aa7-c54e228402a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:24.792696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9075284-e352-4853-8aa7-c54e228402a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:24.793102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9075284-e352-4853-8aa7-c54e228402a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:24.794008 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9075284-e352-4853-8aa7-c54e228402a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:24.796980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9075284-e352-4853-8aa7-c54e228402a5 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:24.836481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 903/1808] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:24.837444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9075284-e352-4853-8aa7-c54e228402a5 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:24.859640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9075284-e352-4853-8aa7-c54e228402a5 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:24.862817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45434f0a-4992-4817-99e7-8013e9bad8e4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:24.877340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7206f0-056a-4ddf-879d-9eeae5d1f8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:24.878174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7206f0-056a-4ddf-879d-9eeae5d1f8ce admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:24.878469 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7206f0-056a-4ddf-879d-9eeae5d1f8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:24.879078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7206f0-056a-4ddf-879d-9eeae5d1f8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:24.879743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7206f0-056a-4ddf-879d-9eeae5d1f8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:24.881421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7206f0-056a-4ddf-879d-9eeae5d1f8ce admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:24.885227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9848ec9-de95-4f73-a916-54786d1b92c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:24.891588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9848ec9-de95-4f73-a916-54786d1b92c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:24.891883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9848ec9-de95-4f73-a916-54786d1b92c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:24.892167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9848ec9-de95-4f73-a916-54786d1b92c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:24.892946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9848ec9-de95-4f73-a916-54786d1b92c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:24.895068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9848ec9-de95-4f73-a916-54786d1b92c5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:24.924659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 904/1809] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:24.924659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7206f0-056a-4ddf-879d-9eeae5d1f8ce admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:24.958985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7206f0-056a-4ddf-879d-9eeae5d1f8ce admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:24.973060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 906/1810] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:24 2022] POST /identity/v3/users => generated 285 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:24.975318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9848ec9-de95-4f73-a916-54786d1b92c5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:24.982728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:24.987936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927b7a5-12cd-4057-ad6a-1a7509be49dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:24.988724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927b7a5-12cd-4057-ad6a-1a7509be49dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:24.989048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927b7a5-12cd-4057-ad6a-1a7509be49dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:24.989337 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927b7a5-12cd-4057-ad6a-1a7509be49dc admin admin] PATH_INFO: `/v3/projects/d0f232245fb741ffb6941da946c551b1/users/3a85ec27e23440a59847825f169b6a1a/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:25.004676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9848ec9-de95-4f73-a916-54786d1b92c5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:25.023095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b7d52-c4b9-44ee-9998-b89ae77a1649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:25.023795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b7d52-c4b9-44ee-9998-b89ae77a1649 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:25.024077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b7d52-c4b9-44ee-9998-b89ae77a1649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:25.024421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b7d52-c4b9-44ee-9998-b89ae77a1649 admin admin] PATH_INFO: `/v3/projects/1c3f443988c1407b8032a8a73e6a16a7/users/008d17d7079e43b7887cfc8792a3f4a2/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:25.027923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927b7a5-12cd-4057-ad6a-1a7509be49dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f232245fb741ffb6941da946c551b1, user_id=3a85ec27e23440a59847825f169b6a1a, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:25.028919 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b7d52-c4b9-44ee-9998-b89ae77a1649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3f443988c1407b8032a8a73e6a16a7, user_id=008d17d7079e43b7887cfc8792a3f4a2, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:25.031630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b7d52-c4b9-44ee-9998-b89ae77a1649 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:25.036701 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927b7a5-12cd-4057-ad6a-1a7509be49dc admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:25.041788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:25.064852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 907/1811] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:24 2022] PUT /identity/v3/projects/1c3f443988c1407b8032a8a73e6a16a7/users/008d17d7079e43b7887cfc8792a3f4a2/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:25.064852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b7d52-c4b9-44ee-9998-b89ae77a1649 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:25.072710 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 905/1812] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:24 2022] PUT /identity/v3/projects/d0f232245fb741ffb6941da946c551b1/users/3a85ec27e23440a59847825f169b6a1a/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:25.074918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158dd6c7-9753-4bce-8264-8353f801b808 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:25.075212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158dd6c7-9753-4bce-8264-8353f801b808 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:25.078116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158dd6c7-9753-4bce-8264-8353f801b808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:25.150412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-158dd6c7-9753-4bce-8264-8353f801b808 None None] MFA Rules not processed for user `008d17d7079e43b7887cfc8792a3f4a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:25.153742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-158dd6c7-9753-4bce-8264-8353f801b808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:25.241101 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b7d52-c4b9-44ee-9998-b89ae77a1649 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:25.323582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 906/1813] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:25.324552 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158dd6c7-9753-4bce-8264-8353f801b808 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:25.361556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17078a1-7c90-4a24-be83-dc71133e7028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:25.362207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17078a1-7c90-4a24-be83-dc71133e7028 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:25.362602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17078a1-7c90-4a24-be83-dc71133e7028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:25.362917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17078a1-7c90-4a24-be83-dc71133e7028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:25.363542 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17078a1-7c90-4a24-be83-dc71133e7028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:25.368692 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17078a1-7c90-4a24-be83-dc71133e7028 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:25.394869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 908/1814] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:25.397593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17078a1-7c90-4a24-be83-dc71133e7028 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:25.404255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a17078a1-7c90-4a24-be83-dc71133e7028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:25.453028 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158dd6c7-9753-4bce-8264-8353f801b808 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:25.475107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e11832-f6d7-4ac0-8171-853be724d0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:25.476196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e11832-f6d7-4ac0-8171-853be724d0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:25.476744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e11832-f6d7-4ac0-8171-853be724d0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:25.477242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e11832-f6d7-4ac0-8171-853be724d0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:25.478118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e11832-f6d7-4ac0-8171-853be724d0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:25.480495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e11832-f6d7-4ac0-8171-853be724d0b3 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:25.493640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 907/1815] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:25.494989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e11832-f6d7-4ac0-8171-853be724d0b3 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:25.507092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e11832-f6d7-4ac0-8171-853be724d0b3 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:25.509840 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17078a1-7c90-4a24-be83-dc71133e7028 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:25.514489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9818ff5-88a0-4b91-b5d7-12d5fdf2a3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:25.515430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9818ff5-88a0-4b91-b5d7-12d5fdf2a3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:25.515915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9818ff5-88a0-4b91-b5d7-12d5fdf2a3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:25.516533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9818ff5-88a0-4b91-b5d7-12d5fdf2a3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:25.517480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9818ff5-88a0-4b91-b5d7-12d5fdf2a3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:25.519527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9818ff5-88a0-4b91-b5d7-12d5fdf2a3f5 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:25.528827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ca845-4fc5-4418-bde5-65dea42eb61d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:25.529711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ca845-4fc5-4418-bde5-65dea42eb61d service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:25.529813 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ca845-4fc5-4418-bde5-65dea42eb61d service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:25.530167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ca845-4fc5-4418-bde5-65dea42eb61d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:25.532104 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-be9ca845-4fc5-4418-bde5-65dea42eb61d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:25.533180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 908/1816] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:25.534177 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9818ff5-88a0-4b91-b5d7-12d5fdf2a3f5 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:25.554164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9818ff5-88a0-4b91-b5d7-12d5fdf2a3f5 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:25.574760 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc64d0d-abd8-4ee9-9533-0ac904d5b2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:25.575793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc64d0d-abd8-4ee9-9533-0ac904d5b2c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:25.576243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc64d0d-abd8-4ee9-9533-0ac904d5b2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:25.576623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc64d0d-abd8-4ee9-9533-0ac904d5b2c0 admin admin] PATH_INFO: `/v3/projects/d0f232245fb741ffb6941da946c551b1/users/3a85ec27e23440a59847825f169b6a1a/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:25.581741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc64d0d-abd8-4ee9-9533-0ac904d5b2c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f232245fb741ffb6941da946c551b1, user_id=3a85ec27e23440a59847825f169b6a1a, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:25.587232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc64d0d-abd8-4ee9-9533-0ac904d5b2c0 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:25.589492 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:25.613748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ca845-4fc5-4418-bde5-65dea42eb61d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:25.615823 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ca845-4fc5-4418-bde5-65dea42eb61d service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:25.618727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 909/1817] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:25 2022] PUT /identity/v3/projects/d0f232245fb741ffb6941da946c551b1/users/3a85ec27e23440a59847825f169b6a1a/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:25.620024 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc64d0d-abd8-4ee9-9533-0ac904d5b2c0 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:25.637595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 909/1818] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:25.642026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f127775-9d6f-4f15-9b15-86c5b6da6d57 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:25.642779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f127775-9d6f-4f15-9b15-86c5b6da6d57 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:25.642879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f127775-9d6f-4f15-9b15-86c5b6da6d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:25.708362 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9f127775-9d6f-4f15-9b15-86c5b6da6d57 None None] MFA Rules not processed for user `3a85ec27e23440a59847825f169b6a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:25.709871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9f127775-9d6f-4f15-9b15-86c5b6da6d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:25.759347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:25.774881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc64d0d-abd8-4ee9-9533-0ac904d5b2c0 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:25.797096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce9224-504b-4924-a965-9adc08988b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:25.797921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce9224-504b-4924-a965-9adc08988b85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:25.798294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce9224-504b-4924-a965-9adc08988b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:25.798758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce9224-504b-4924-a965-9adc08988b85 admin admin] PATH_INFO: `/v3/projects/27370d4c7f7e4f50bec1373329d2832c/users/2544c12e442144c482c8be0bf2486244/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:25.830912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce9224-504b-4924-a965-9adc08988b85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27370d4c7f7e4f50bec1373329d2832c, user_id=2544c12e442144c482c8be0bf2486244, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:25.834245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce9224-504b-4924-a965-9adc08988b85 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:25.853392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 910/1819] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:25.865862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f127775-9d6f-4f15-9b15-86c5b6da6d57 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:25.872974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9f127775-9d6f-4f15-9b15-86c5b6da6d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:25.912972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 910/1820] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:25 2022] PUT /identity/v3/projects/27370d4c7f7e4f50bec1373329d2832c/users/2544c12e442144c482c8be0bf2486244/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:25.923975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51a048-49ca-442e-b0b7-b56c746d6dd0 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:25.925499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51a048-49ca-442e-b0b7-b56c746d6dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:25.925788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51a048-49ca-442e-b0b7-b56c746d6dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:26.030944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-5c51a048-49ca-442e-b0b7-b56c746d6dd0 None None] MFA Rules not processed for user `2544c12e442144c482c8be0bf2486244`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:26.032630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5c51a048-49ca-442e-b0b7-b56c746d6dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:26.076279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f127775-9d6f-4f15-9b15-86c5b6da6d57 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:26.178269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 911/1821] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:26.178269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec23087-5a66-4929-b2ae-38cb2b17e199 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:26.178269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec23087-5a66-4929-b2ae-38cb2b17e199 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:26.178269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec23087-5a66-4929-b2ae-38cb2b17e199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:26.217969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3ec23087-5a66-4929-b2ae-38cb2b17e199 None None] MFA Rules not processed for user `9e0b7efc488a42ffab8ba7e2d101ced4`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:26.219324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3ec23087-5a66-4929-b2ae-38cb2b17e199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:26.263024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b813e7a-7872-4950-be77-e0f2b7c98894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:26.263820 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b813e7a-7872-4950-be77-e0f2b7c98894 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:26.265541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b813e7a-7872-4950-be77-e0f2b7c98894 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:26.269466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b813e7a-7872-4950-be77-e0f2b7c98894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:26.352275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:26.354136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 912/1822] 10.222.0.58 () {58 vars in 1015 bytes} [Fri Aug 19 16:32:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:26.355198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec23087-5a66-4929-b2ae-38cb2b17e199 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:26.421774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b813e7a-7872-4950-be77-e0f2b7c98894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:26.423335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b813e7a-7872-4950-be77-e0f2b7c98894 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:26.455816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 911/1823] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:26.474484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b813e7a-7872-4950-be77-e0f2b7c98894 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:26.476093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6b813e7a-7872-4950-be77-e0f2b7c98894 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:26.490840 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b813e7a-7872-4950-be77-e0f2b7c98894 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:26.495007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52efc-bb41-4c5d-9319-1dacda7059f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:26.495721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52efc-bb41-4c5d-9319-1dacda7059f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:26.496207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52efc-bb41-4c5d-9319-1dacda7059f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:26.496612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52efc-bb41-4c5d-9319-1dacda7059f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:26.507690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa52efc-bb41-4c5d-9319-1dacda7059f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:26.509660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa52efc-bb41-4c5d-9319-1dacda7059f2 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:26.522642 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 912/1824] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:26.536375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec23087-5a66-4929-b2ae-38cb2b17e199 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:26.563023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2982d2-a4ee-4c68-b7a3-bb1a89e2ffc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:26.563974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2982d2-a4ee-4c68-b7a3-bb1a89e2ffc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:26.564607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2982d2-a4ee-4c68-b7a3-bb1a89e2ffc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:26.565446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2982d2-a4ee-4c68-b7a3-bb1a89e2ffc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:26.566310 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2982d2-a4ee-4c68-b7a3-bb1a89e2ffc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:26.568903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2982d2-a4ee-4c68-b7a3-bb1a89e2ffc4 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:26.609055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:26.622558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 913/1825] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:26 2022] POST /identity/v3/projects => generated 367 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:26.623853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52efc-bb41-4c5d-9319-1dacda7059f2 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:26.631484 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2982d2-a4ee-4c68-b7a3-bb1a89e2ffc4 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:26.642033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52efc-bb41-4c5d-9319-1dacda7059f2 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:26.712602 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2982d2-a4ee-4c68-b7a3-bb1a89e2ffc4 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:26.721926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b0ae9-624d-4c0d-be5d-5b432a50de27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:26.722706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b0ae9-624d-4c0d-be5d-5b432a50de27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:26.723903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b0ae9-624d-4c0d-be5d-5b432a50de27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:26.724357 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b0ae9-624d-4c0d-be5d-5b432a50de27 admin admin] PATH_INFO: `/v3/users/2984bb939db0458cb9040c57d2f1bbf3` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:26.726080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b0ae9-624d-4c0d-be5d-5b432a50de27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2984bb939db0458cb9040c57d2f1bbf3)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:26.728225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b0ae9-624d-4c0d-be5d-5b432a50de27 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:26.757888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d837d-d91e-4f79-8c64-4c2460e4c29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:26.759188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d837d-d91e-4f79-8c64-4c2460e4c29d admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:26.760497 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d837d-d91e-4f79-8c64-4c2460e4c29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:26.761484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d837d-d91e-4f79-8c64-4c2460e4c29d admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:26.764586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d837d-d91e-4f79-8c64-4c2460e4c29d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:26.771385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d837d-d91e-4f79-8c64-4c2460e4c29d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:26.820818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 913/1826] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:26 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:26.823609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c08b0ae9-624d-4c0d-be5d-5b432a50de27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2984bb939db0458cb9040c57d2f1bbf3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:26.824396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c08b0ae9-624d-4c0d-be5d-5b432a50de27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2984bb939db0458cb9040c57d2f1bbf3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:26.825120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c08b0ae9-624d-4c0d-be5d-5b432a50de27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2984bb939db0458cb9040c57d2f1bbf3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:26.826592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d837d-d91e-4f79-8c64-4c2460e4c29d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:26.829948 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-143d837d-d91e-4f79-8c64-4c2460e4c29d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:26.844512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c08b0ae9-624d-4c0d-be5d-5b432a50de27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2984bb939db0458cb9040c57d2f1bbf3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:26.872691 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d837d-d91e-4f79-8c64-4c2460e4c29d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:26.884985 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e20975-ad34-41ca-a154-e8c6de661a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:26.885934 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e20975-ad34-41ca-a154-e8c6de661a1a admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:26.886086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c08b0ae9-624d-4c0d-be5d-5b432a50de27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2984bb939db0458cb9040c57d2f1bbf3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:26.886363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e20975-ad34-41ca-a154-e8c6de661a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:26.886706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e20975-ad34-41ca-a154-e8c6de661a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:26.887441 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e20975-ad34-41ca-a154-e8c6de661a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:26.889815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e20975-ad34-41ca-a154-e8c6de661a1a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:26.915347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 914/1827] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:26.917359 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e20975-ad34-41ca-a154-e8c6de661a1a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:26.922338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 914/1828] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:26 2022] DELETE /identity/v3/users/2984bb939db0458cb9040c57d2f1bbf3 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:26.927005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc60179-4df7-4774-8efc-63e14f86f1a7 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:26.927289 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc60179-4df7-4774-8efc-63e14f86f1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:26.927686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc60179-4df7-4774-8efc-63e14f86f1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:26.935040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e20975-ad34-41ca-a154-e8c6de661a1a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:26.950563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-0dc60179-4df7-4774-8efc-63e14f86f1a7 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:26.951650 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0dc60179-4df7-4774-8efc-63e14f86f1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:27.037906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8e4a1-d19f-4b01-80c0-7e04c0266ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:27.038602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8e4a1-d19f-4b01-80c0-7e04c0266ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:27.039445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8e4a1-d19f-4b01-80c0-7e04c0266ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:27.039445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8e4a1-d19f-4b01-80c0-7e04c0266ad3 admin admin] PATH_INFO: `/v3/projects/a22961157f1b4b84927eb3e1424d9d9f/users/0d7ad47ab10143e3a99f4088e58d4d50/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:27.066477 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8e4a1-d19f-4b01-80c0-7e04c0266ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22961157f1b4b84927eb3e1424d9d9f, user_id=0d7ad47ab10143e3a99f4088e58d4d50, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:27.069363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8e4a1-d19f-4b01-80c0-7e04c0266ad3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:27.084827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 915/1829] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:27.097094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc60179-4df7-4774-8efc-63e14f86f1a7 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:27.098942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0dc60179-4df7-4774-8efc-63e14f86f1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:27.116845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 915/1830] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:26 2022] PUT /identity/v3/projects/a22961157f1b4b84927eb3e1424d9d9f/users/0d7ad47ab10143e3a99f4088e58d4d50/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:27.120387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8e4a1-d19f-4b01-80c0-7e04c0266ad3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:27.135426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:27.162894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:27.199826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc60179-4df7-4774-8efc-63e14f86f1a7 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:27.203862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8e4a1-d19f-4b01-80c0-7e04c0266ad3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:27.209312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfb8c7-ceef-4244-86f8-4617ad593346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:27.210371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfb8c7-ceef-4244-86f8-4617ad593346 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:27.210954 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfb8c7-ceef-4244-86f8-4617ad593346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:27.211382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfb8c7-ceef-4244-86f8-4617ad593346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:27.212179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfb8c7-ceef-4244-86f8-4617ad593346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:27.214125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfb8c7-ceef-4244-86f8-4617ad593346 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:27.230122 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 916/1831] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:27.231558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfb8c7-ceef-4244-86f8-4617ad593346 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:27.243621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfb8c7-ceef-4244-86f8-4617ad593346 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:27.250580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe42686-f0e7-406e-97a3-0c83eb765ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:27.251668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe42686-f0e7-406e-97a3-0c83eb765ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:27.252159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe42686-f0e7-406e-97a3-0c83eb765ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:27.252439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe42686-f0e7-406e-97a3-0c83eb765ed9 admin admin] PATH_INFO: `/v3/projects/a22961157f1b4b84927eb3e1424d9d9f/users/0d7ad47ab10143e3a99f4088e58d4d50/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:27.254587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe42686-f0e7-406e-97a3-0c83eb765ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22961157f1b4b84927eb3e1424d9d9f, user_id=0d7ad47ab10143e3a99f4088e58d4d50, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:27.257321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe42686-f0e7-406e-97a3-0c83eb765ed9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:27.272953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 917/1832] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:27 2022] PUT /identity/v3/projects/a22961157f1b4b84927eb3e1424d9d9f/users/0d7ad47ab10143e3a99f4088e58d4d50/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:27.281482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a1c1f-fee8-4038-b70a-12156ced7b10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:27.284736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a1c1f-fee8-4038-b70a-12156ced7b10 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:27.286649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d6053-2c34-4b4a-821d-5f56c7fae826 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:27.286649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d6053-2c34-4b4a-821d-5f56c7fae826 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:27.286649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d6053-2c34-4b4a-821d-5f56c7fae826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:27.287637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a1c1f-fee8-4038-b70a-12156ced7b10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:27.290133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a1c1f-fee8-4038-b70a-12156ced7b10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:27.308105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a1c1f-fee8-4038-b70a-12156ced7b10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:27.309767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a1c1f-fee8-4038-b70a-12156ced7b10 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:27.325263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-353d6053-2c34-4b4a-821d-5f56c7fae826 None None] MFA Rules not processed for user `0d7ad47ab10143e3a99f4088e58d4d50`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:27.326517 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-353d6053-2c34-4b4a-821d-5f56c7fae826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:27.328062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 916/1833] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:27.437637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 918/1834] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:27.457651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a1c1f-fee8-4038-b70a-12156ced7b10 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:27.460006 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-910a1c1f-fee8-4038-b70a-12156ced7b10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:27.557688 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a1c1f-fee8-4038-b70a-12156ced7b10 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:27.582675 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d6053-2c34-4b4a-821d-5f56c7fae826 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:27.584397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-353d6053-2c34-4b4a-821d-5f56c7fae826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:27.649178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eba35d-f36d-42a4-800b-8ae2e20708eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:27.650059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eba35d-f36d-42a4-800b-8ae2e20708eb service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:27.651834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eba35d-f36d-42a4-800b-8ae2e20708eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:27.652354 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eba35d-f36d-42a4-800b-8ae2e20708eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:27.678206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d6053-2c34-4b4a-821d-5f56c7fae826 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:27.678839 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eba35d-f36d-42a4-800b-8ae2e20708eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:27.681326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eba35d-f36d-42a4-800b-8ae2e20708eb service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:27.682216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8746c-dbbb-4789-9f75-2fe889ece956 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:27.683060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8746c-dbbb-4789-9f75-2fe889ece956 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:27.683341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8746c-dbbb-4789-9f75-2fe889ece956 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:27.683617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8746c-dbbb-4789-9f75-2fe889ece956 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:27.685619 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9cc8746c-dbbb-4789-9f75-2fe889ece956 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:27.694810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 917/1835] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:27.758021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:27.774622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8746c-dbbb-4789-9f75-2fe889ece956 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:27.777646 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8746c-dbbb-4789-9f75-2fe889ece956 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:27.792624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 919/1836] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:29.018330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eba35d-f36d-42a4-800b-8ae2e20708eb service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:29.028073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eba35d-f36d-42a4-800b-8ae2e20708eb service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:29.033347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb58c9-f59c-4954-b9ce-8ac50076d482 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:29.035571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb58c9-f59c-4954-b9ce-8ac50076d482 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:29.036324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb58c9-f59c-4954-b9ce-8ac50076d482 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:29.036820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb58c9-f59c-4954-b9ce-8ac50076d482 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:29.040339 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ffeb58c9-f59c-4954-b9ce-8ac50076d482 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:29.088155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:29.160014 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb58c9-f59c-4954-b9ce-8ac50076d482 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:29.163793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb58c9-f59c-4954-b9ce-8ac50076d482 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:29.189636 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 918/1837] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:29.606474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8746c-dbbb-4789-9f75-2fe889ece956 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:29.608076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9cc8746c-dbbb-4789-9f75-2fe889ece956 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:29.724532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8746c-dbbb-4789-9f75-2fe889ece956 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:29.733567 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:29.734553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:29.734975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:29.735356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9 admin admin] PATH_INFO: `/v3/projects/3960897483904b508688ce79adfde4ca` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:29.738366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3960897483904b508688ce79adfde4ca)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:29.741794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:29.842642 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3960897483904b508688ce79adfde4ca', 'request_id': 'req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:29.849469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3960897483904b508688ce79adfde4ca', 'request_id': 'req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:29.860463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3960897483904b508688ce79adfde4ca', 'request_id': 'req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:29.881061 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9 admin admin] The token cache is being invalidate because project 3960897483904b508688ce79adfde4ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:29.881401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:29.883999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7ae1495-f07a-4c3f-bd79-de560f49e2a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:29.889106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 920/1838] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:29 2022] DELETE /identity/v3/projects/3960897483904b508688ce79adfde4ca => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:29.892207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb58c9-f59c-4954-b9ce-8ac50076d482 service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:29.907480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb58c9-f59c-4954-b9ce-8ac50076d482 service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:30.009315 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4c3ed-e2bc-4666-b09a-dae9f05d3fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:30.010865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4c3ed-e2bc-4666-b09a-dae9f05d3fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:30.013190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4c3ed-e2bc-4666-b09a-dae9f05d3fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:30.013424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4c3ed-e2bc-4666-b09a-dae9f05d3fb2 admin admin] PATH_INFO: `/v3/users/3a85ec27e23440a59847825f169b6a1a` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:30.015438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4c3ed-e2bc-4666-b09a-dae9f05d3fb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a85ec27e23440a59847825f169b6a1a)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:30.018235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4c3ed-e2bc-4666-b09a-dae9f05d3fb2 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:30.138695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-55d4c3ed-e2bc-4666-b09a-dae9f05d3fb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a85ec27e23440a59847825f169b6a1a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:30.139383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-55d4c3ed-e2bc-4666-b09a-dae9f05d3fb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a85ec27e23440a59847825f169b6a1a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:30.140155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-55d4c3ed-e2bc-4666-b09a-dae9f05d3fb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a85ec27e23440a59847825f169b6a1a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:30.156001 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-55d4c3ed-e2bc-4666-b09a-dae9f05d3fb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a85ec27e23440a59847825f169b6a1a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:30.163540 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba55644-03e8-4007-8563-e727137a3d20 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:30.163988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba55644-03e8-4007-8563-e727137a3d20 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:30.164269 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba55644-03e8-4007-8563-e727137a3d20 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:30.166754 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 921/1839] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:32:30.187660 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-55d4c3ed-e2bc-4666-b09a-dae9f05d3fb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a85ec27e23440a59847825f169b6a1a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:30.213611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 919/1840] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:29 2022] DELETE /identity/v3/users/3a85ec27e23440a59847825f169b6a1a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:30.231650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba55644-03e8-4007-8563-e727137a3d20 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:30.233430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6ba55644-03e8-4007-8563-e727137a3d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:30.241390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c27ca-96eb-44bf-acba-602166cdb4d0 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:30.241695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c27ca-96eb-44bf-acba-602166cdb4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:30.241974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c27ca-96eb-44bf-acba-602166cdb4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:30.283654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-c68c27ca-96eb-44bf-acba-602166cdb4d0 None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:30.285658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c68c27ca-96eb-44bf-acba-602166cdb4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:30.423995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:30.487091 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 920/1841] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:30.559110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3e660-11d5-4b47-b850-0c1da169e80d None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:30.559960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3e660-11d5-4b47-b850-0c1da169e80d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:30.559960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3e660-11d5-4b47-b850-0c1da169e80d None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:30.561055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 921/1842] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 16:32:30.574564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba55644-03e8-4007-8563-e727137a3d20 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:30.581415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:30.582082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:30.582417 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:30.582857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] PATH_INFO: `/v3/users/2544c12e442144c482c8be0bf2486244` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:30.584190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2544c12e442144c482c8be0bf2486244)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:30.588373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:30.602720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf51b4-7d14-4488-a345-76660d1b134c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:30.603026 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf51b4-7d14-4488-a345-76660d1b134c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:30.603359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf51b4-7d14-4488-a345-76660d1b134c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:30.643144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3cbf51b4-7d14-4488-a345-76660d1b134c None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:30.644387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3cbf51b4-7d14-4488-a345-76660d1b134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:30.662683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:30.670391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 922/1843] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:30.725045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2544c12e442144c482c8be0bf2486244'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:30.726674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2544c12e442144c482c8be0bf2486244'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:30.728158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2544c12e442144c482c8be0bf2486244'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:30.748415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2544c12e442144c482c8be0bf2486244'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:30.759327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2544c12e442144c482c8be0bf2486244'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:30.787566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 922/1844] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:30 2022] DELETE /identity/v3/users/2544c12e442144c482c8be0bf2486244 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:30.791206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621827a-f588-4dc1-9a5e-bc142e4fede7 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:30.791590 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621827a-f588-4dc1-9a5e-bc142e4fede7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:30.792080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621827a-f588-4dc1-9a5e-bc142e4fede7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:30.821225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-8621827a-f588-4dc1-9a5e-bc142e4fede7 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:30.824008 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8621827a-f588-4dc1-9a5e-bc142e4fede7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:30.925901 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 923/1845] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:30.967842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:30.969448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:31.080449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990190b8-326c-4f75-b7fe-f0e9beb2d02f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:31.085537 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c3b3f-a118-4b0e-9044-149048a7b176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:31.086889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c3b3f-a118-4b0e-9044-149048a7b176 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:31.087389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c3b3f-a118-4b0e-9044-149048a7b176 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:31.087898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c3b3f-a118-4b0e-9044-149048a7b176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:31.099580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c3b3f-a118-4b0e-9044-149048a7b176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:31.101480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c3b3f-a118-4b0e-9044-149048a7b176 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:31.179916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 923/1846] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:31.676690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621827a-f588-4dc1-9a5e-bc142e4fede7 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:31.678381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8621827a-f588-4dc1-9a5e-bc142e4fede7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:31.779148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621827a-f588-4dc1-9a5e-bc142e4fede7 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:31.786882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b84ec-cd9e-48ca-96cd-cdff407fa4b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:31.787611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b84ec-cd9e-48ca-96cd-cdff407fa4b7 service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:31.787956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b84ec-cd9e-48ca-96cd-cdff407fa4b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:31.788609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b84ec-cd9e-48ca-96cd-cdff407fa4b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:31.790316 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-be8b84ec-cd9e-48ca-96cd-cdff407fa4b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:31.804537 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b84ec-cd9e-48ca-96cd-cdff407fa4b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:31.806309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b84ec-cd9e-48ca-96cd-cdff407fa4b7 service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:31.834754 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 924/1847] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:31.911143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c3b3f-a118-4b0e-9044-149048a7b176 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:31.934658 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c3b3f-a118-4b0e-9044-149048a7b176 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:31.946378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49762bcc-7a59-4bc0-b273-22eb2b3d08c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:31.947074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49762bcc-7a59-4bc0-b273-22eb2b3d08c8 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:31.947448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49762bcc-7a59-4bc0-b273-22eb2b3d08c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:31.948197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49762bcc-7a59-4bc0-b273-22eb2b3d08c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:31.950197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-49762bcc-7a59-4bc0-b273-22eb2b3d08c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:31.973129 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49762bcc-7a59-4bc0-b273-22eb2b3d08c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:31.975461 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49762bcc-7a59-4bc0-b273-22eb2b3d08c8 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:31.996460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 924/1848] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:32.530554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b84ec-cd9e-48ca-96cd-cdff407fa4b7 service nova] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:32.531994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-be8b84ec-cd9e-48ca-96cd-cdff407fa4b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:32.562255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b84ec-cd9e-48ca-96cd-cdff407fa4b7 service nova] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:32.571310 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571229fa-fbc1-43e1-b338-8a0c95932189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:32.572097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571229fa-fbc1-43e1-b338-8a0c95932189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:32.575663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571229fa-fbc1-43e1-b338-8a0c95932189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:32.575663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571229fa-fbc1-43e1-b338-8a0c95932189 admin admin] PATH_INFO: `/v3/projects/d0f232245fb741ffb6941da946c551b1` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:32.587419 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571229fa-fbc1-43e1-b338-8a0c95932189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f232245fb741ffb6941da946c551b1)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:32.590114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571229fa-fbc1-43e1-b338-8a0c95932189 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:32.707257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-571229fa-fbc1-43e1-b338-8a0c95932189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f232245fb741ffb6941da946c551b1', 'request_id': 'req-571229fa-fbc1-43e1-b338-8a0c95932189'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:32.719763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-571229fa-fbc1-43e1-b338-8a0c95932189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f232245fb741ffb6941da946c551b1', 'request_id': 'req-571229fa-fbc1-43e1-b338-8a0c95932189'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:32.719763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:32.727281 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-571229fa-fbc1-43e1-b338-8a0c95932189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f232245fb741ffb6941da946c551b1', 'request_id': 'req-571229fa-fbc1-43e1-b338-8a0c95932189'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:32.743231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-571229fa-fbc1-43e1-b338-8a0c95932189 admin admin] The token cache is being invalidate because project d0f232245fb741ffb6941da946c551b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:32.743693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-571229fa-fbc1-43e1-b338-8a0c95932189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:32.745569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-571229fa-fbc1-43e1-b338-8a0c95932189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:32.748791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 925/1849] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:32 2022] DELETE /identity/v3/projects/d0f232245fb741ffb6941da946c551b1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:32.753868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ca498-868b-42f6-b78e-972c468087ab None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:32.754143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ca498-868b-42f6-b78e-972c468087ab None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:32.755143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ca498-868b-42f6-b78e-972c468087ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:32.780526 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7d6ca498-868b-42f6-b78e-972c468087ab None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:32.781800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7d6ca498-868b-42f6-b78e-972c468087ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:32.867554 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 925/1850] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:32.875501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a01def-e9cd-45ee-8be2-375b2b06908c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:32.876054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a01def-e9cd-45ee-8be2-375b2b06908c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:32.876493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a01def-e9cd-45ee-8be2-375b2b06908c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:32.919318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d3a01def-e9cd-45ee-8be2-375b2b06908c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:32.921010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d3a01def-e9cd-45ee-8be2-375b2b06908c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:32.945145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ca498-868b-42f6-b78e-972c468087ab None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:32.948846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 926/1851] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:32.949932 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a01def-e9cd-45ee-8be2-375b2b06908c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:32.960265 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ca498-868b-42f6-b78e-972c468087ab None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:32.965402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d3a01def-e9cd-45ee-8be2-375b2b06908c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:32.966916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6633208-e816-4b69-9bbb-f340814770b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:32.967827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6633208-e816-4b69-9bbb-f340814770b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:32.968324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6633208-e816-4b69-9bbb-f340814770b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:32.969531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6633208-e816-4b69-9bbb-f340814770b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:32.970390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6633208-e816-4b69-9bbb-f340814770b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:32.971087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6633208-e816-4b69-9bbb-f340814770b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:32.975475 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6633208-e816-4b69-9bbb-f340814770b3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:32.984415 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a01def-e9cd-45ee-8be2-375b2b06908c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:32.990391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:32.991151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:32.991565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 926/1852] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:32:32.991687 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:32.992026 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] PATH_INFO: `/v3/projects/27370d4c7f7e4f50bec1373329d2832c` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:32.993251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=27370d4c7f7e4f50bec1373329d2832c)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:32.995559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:32.995790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6633208-e816-4b69-9bbb-f340814770b3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:33.011789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6633208-e816-4b69-9bbb-f340814770b3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:33.019085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d77bf2-3492-424c-9826-bb07a8d48380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:33.019981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d77bf2-3492-424c-9826-bb07a8d48380 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.020451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d77bf2-3492-424c-9826-bb07a8d48380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.020979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d77bf2-3492-424c-9826-bb07a8d48380 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.021711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d77bf2-3492-424c-9826-bb07a8d48380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:33.024053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d77bf2-3492-424c-9826-bb07a8d48380 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:33.044868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:33.063286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 927/1853] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:32 2022] POST /identity/v3/projects => generated 367 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:32:33.066466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d77bf2-3492-424c-9826-bb07a8d48380 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:33.071319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27370d4c7f7e4f50bec1373329d2832c', 'request_id': 'req-880c8724-e889-4785-a6cd-4b8c680e22bd'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.078223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27370d4c7f7e4f50bec1373329d2832c', 'request_id': 'req-880c8724-e889-4785-a6cd-4b8c680e22bd'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.083698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27370d4c7f7e4f50bec1373329d2832c', 'request_id': 'req-880c8724-e889-4785-a6cd-4b8c680e22bd'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.096664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] The token cache is being invalidate because project 27370d4c7f7e4f50bec1373329d2832c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:33.097008 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.097650 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.099403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 927/1854] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:32 2022] DELETE /identity/v3/projects/27370d4c7f7e4f50bec1373329d2832c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:33.103203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:33.105096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:33.132558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d77bf2-3492-424c-9826-bb07a8d48380 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:33.137814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fc400-880d-4acf-a9f3-1aefa9714387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:33.138427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fc400-880d-4acf-a9f3-1aefa9714387 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.138745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fc400-880d-4acf-a9f3-1aefa9714387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.139013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fc400-880d-4acf-a9f3-1aefa9714387 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.139637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0fc400-880d-4acf-a9f3-1aefa9714387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:33.141593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0fc400-880d-4acf-a9f3-1aefa9714387 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:33.165353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 928/1855] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:33 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:32:33.170207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fc400-880d-4acf-a9f3-1aefa9714387 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:33.171915 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1c0fc400-880d-4acf-a9f3-1aefa9714387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:33.178860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8724-e889-4785-a6cd-4b8c680e22bd admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:33.184716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fc400-880d-4acf-a9f3-1aefa9714387 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:33.186858 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c19a2a-f91f-43a2-a125-1e0928c9a4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:33.187769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c19a2a-f91f-43a2-a125-1e0928c9a4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.188050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c19a2a-f91f-43a2-a125-1e0928c9a4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.188323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c19a2a-f91f-43a2-a125-1e0928c9a4fb admin admin] PATH_INFO: `/v3/users/0d7ad47ab10143e3a99f4088e58d4d50` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.189485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c19a2a-f91f-43a2-a125-1e0928c9a4fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7ad47ab10143e3a99f4088e58d4d50)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:33.189660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794ca73-fab7-4155-b19c-23075dd8b282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:33.190261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794ca73-fab7-4155-b19c-23075dd8b282 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.190538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794ca73-fab7-4155-b19c-23075dd8b282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.190808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794ca73-fab7-4155-b19c-23075dd8b282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.191402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c19a2a-f91f-43a2-a125-1e0928c9a4fb admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:33.191501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794ca73-fab7-4155-b19c-23075dd8b282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:33.193033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794ca73-fab7-4155-b19c-23075dd8b282 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:33.204597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 929/1856] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:33.208608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794ca73-fab7-4155-b19c-23075dd8b282 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:33.219880 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794ca73-fab7-4155-b19c-23075dd8b282 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:33.225340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9633ac-5e13-48dc-842b-7f4fbfc7eabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:33.226103 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9633ac-5e13-48dc-842b-7f4fbfc7eabd admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.226385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9633ac-5e13-48dc-842b-7f4fbfc7eabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.226683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9633ac-5e13-48dc-842b-7f4fbfc7eabd admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.227280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9633ac-5e13-48dc-842b-7f4fbfc7eabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:33.230028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9633ac-5e13-48dc-842b-7f4fbfc7eabd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:33.246272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 930/1857] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:33.248118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e0c19a2a-f91f-43a2-a125-1e0928c9a4fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7ad47ab10143e3a99f4088e58d4d50'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.248813 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e0c19a2a-f91f-43a2-a125-1e0928c9a4fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7ad47ab10143e3a99f4088e58d4d50'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.249453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e0c19a2a-f91f-43a2-a125-1e0928c9a4fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7ad47ab10143e3a99f4088e58d4d50'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.250101 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9633ac-5e13-48dc-842b-7f4fbfc7eabd admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:33.252463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9e9633ac-5e13-48dc-842b-7f4fbfc7eabd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:33.256720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e0c19a2a-f91f-43a2-a125-1e0928c9a4fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7ad47ab10143e3a99f4088e58d4d50'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.264572 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:33.266367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9633ac-5e13-48dc-842b-7f4fbfc7eabd admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:33.271664 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b2717-932e-49ff-88dd-3ee55b2dfb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:33.272381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b2717-932e-49ff-88dd-3ee55b2dfb1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.272923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b2717-932e-49ff-88dd-3ee55b2dfb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.273244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b2717-932e-49ff-88dd-3ee55b2dfb1f admin admin] PATH_INFO: `/v3/projects/eff36f36cf69438ea25483a680ba0a28/users/5d56eeaf459640b6b81cab1a27c9cffe/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.273775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e0c19a2a-f91f-43a2-a125-1e0928c9a4fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7ad47ab10143e3a99f4088e58d4d50'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.292121 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b2717-932e-49ff-88dd-3ee55b2dfb1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff36f36cf69438ea25483a680ba0a28, user_id=5d56eeaf459640b6b81cab1a27c9cffe, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:33.292968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 928/1858] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:33 2022] DELETE /identity/v3/users/0d7ad47ab10143e3a99f4088e58d4d50 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:33.294918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b2717-932e-49ff-88dd-3ee55b2dfb1f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:33.320485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 931/1859] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:33 2022] PUT /identity/v3/projects/eff36f36cf69438ea25483a680ba0a28/users/5d56eeaf459640b6b81cab1a27c9cffe/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:33.331645 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e7c29-a85c-45cf-a21c-63eab823ca73 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.332014 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e7c29-a85c-45cf-a21c-63eab823ca73 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.332380 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e7c29-a85c-45cf-a21c-63eab823ca73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.368274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-742e7c29-a85c-45cf-a21c-63eab823ca73 None None] MFA Rules not processed for user `5d56eeaf459640b6b81cab1a27c9cffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:33.370125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-742e7c29-a85c-45cf-a21c-63eab823ca73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:33.442404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 929/1860] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:33.446884 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b2717-932e-49ff-88dd-3ee55b2dfb1f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:33.517170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b2717-932e-49ff-88dd-3ee55b2dfb1f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:33.521871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cad162-b6d6-4991-a83f-9776b8488c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:33.522474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cad162-b6d6-4991-a83f-9776b8488c49 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.522754 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cad162-b6d6-4991-a83f-9776b8488c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.523030 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cad162-b6d6-4991-a83f-9776b8488c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.523663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cad162-b6d6-4991-a83f-9776b8488c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:33.526163 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cad162-b6d6-4991-a83f-9776b8488c49 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:33.553675 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 932/1861] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:33 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:33.558353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e7c29-a85c-45cf-a21c-63eab823ca73 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:33.569745 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e7c29-a85c-45cf-a21c-63eab823ca73 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:33.591350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cad162-b6d6-4991-a83f-9776b8488c49 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:33.593371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-24cad162-b6d6-4991-a83f-9776b8488c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:33.625539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:33.651856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d69ba8-a221-4037-9532-974d566cc7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:33.652478 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d69ba8-a221-4037-9532-974d566cc7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.652889 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d69ba8-a221-4037-9532-974d566cc7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.653121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d69ba8-a221-4037-9532-974d566cc7e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.653704 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d69ba8-a221-4037-9532-974d566cc7e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:33.655664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d69ba8-a221-4037-9532-974d566cc7e1 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:33.667849 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cad162-b6d6-4991-a83f-9776b8488c49 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:33.672996 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:33.673633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.673912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.674189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] PATH_INFO: `/v3/projects/a22961157f1b4b84927eb3e1424d9d9f` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.675305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a22961157f1b4b84927eb3e1424d9d9f)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:33.677240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:33.681063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 930/1862] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:33 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:33.682228 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d69ba8-a221-4037-9532-974d566cc7e1 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:33.692551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d69ba8-a221-4037-9532-974d566cc7e1 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:33.699378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1e203-62f8-428f-abe3-58e0b4db37db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:33.700401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1e203-62f8-428f-abe3-58e0b4db37db admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.700401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1e203-62f8-428f-abe3-58e0b4db37db admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.700538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1e203-62f8-428f-abe3-58e0b4db37db admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.701184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b1e203-62f8-428f-abe3-58e0b4db37db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:33.702924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b1e203-62f8-428f-abe3-58e0b4db37db admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:33.717847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 931/1863] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:32:33.727930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1e203-62f8-428f-abe3-58e0b4db37db admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:33.749827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1e203-62f8-428f-abe3-58e0b4db37db admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:33.773250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a22961157f1b4b84927eb3e1424d9d9f', 'request_id': 'req-60aade2a-20e9-4958-88bb-50959b96f930'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.790963 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a22961157f1b4b84927eb3e1424d9d9f', 'request_id': 'req-60aade2a-20e9-4958-88bb-50959b96f930'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.803363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a22961157f1b4b84927eb3e1424d9d9f', 'request_id': 'req-60aade2a-20e9-4958-88bb-50959b96f930'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.829912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] The token cache is being invalidate because project a22961157f1b4b84927eb3e1424d9d9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:33.831144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.833054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:33.842332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 933/1864] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:33 2022] DELETE /identity/v3/projects/a22961157f1b4b84927eb3e1424d9d9f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:33.843701 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:33.845639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:33.850321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:33.862528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59fcaad-e6a7-4bee-b9c2-6806d091c2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:33.863208 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fcaad-e6a7-4bee-b9c2-6806d091c2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.863490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fcaad-e6a7-4bee-b9c2-6806d091c2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.863767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fcaad-e6a7-4bee-b9c2-6806d091c2e0 admin admin] PATH_INFO: `/v3/projects/fbbe037cacbb4a0d8925699eb6b5df2c/users/f2f9e01cf62047a4b32671960e0d7677/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.882554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59fcaad-e6a7-4bee-b9c2-6806d091c2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbbe037cacbb4a0d8925699eb6b5df2c, user_id=f2f9e01cf62047a4b32671960e0d7677, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:33.887116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59fcaad-e6a7-4bee-b9c2-6806d091c2e0 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:33.910813 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 932/1865] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:33 2022] PUT /identity/v3/projects/fbbe037cacbb4a0d8925699eb6b5df2c/users/f2f9e01cf62047a4b32671960e0d7677/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:33.912591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b41917-a20b-4a76-9d9a-538b42402f71 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:33.912999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b41917-a20b-4a76-9d9a-538b42402f71 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:33.913376 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b41917-a20b-4a76-9d9a-538b42402f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:33.938598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aade2a-20e9-4958-88bb-50959b96f930 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:33.950669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-01b41917-a20b-4a76-9d9a-538b42402f71 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:33.952921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-01b41917-a20b-4a76-9d9a-538b42402f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:34.021900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ae0d6-aae3-471c-9513-b1ffd4f7df81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:34.022556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ae0d6-aae3-471c-9513-b1ffd4f7df81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.022836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ae0d6-aae3-471c-9513-b1ffd4f7df81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.023110 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ae0d6-aae3-471c-9513-b1ffd4f7df81 admin admin] PATH_INFO: `/v3/users/619e3cb8198c460db8bf8292fb123597` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.024406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ae0d6-aae3-471c-9513-b1ffd4f7df81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=619e3cb8198c460db8bf8292fb123597)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:34.028207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ae0d6-aae3-471c-9513-b1ffd4f7df81 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:34.030687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:34.048907 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 933/1866] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:34.049311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b41917-a20b-4a76-9d9a-538b42402f71 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:34.050558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-01b41917-a20b-4a76-9d9a-538b42402f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:34.066520 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b41917-a20b-4a76-9d9a-538b42402f71 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:34.070371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bb21d-4ddb-4a6c-9dc6-951b169b7889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:34.071096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bb21d-4ddb-4a6c-9dc6-951b169b7889 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.071472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bb21d-4ddb-4a6c-9dc6-951b169b7889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.072003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bb21d-4ddb-4a6c-9dc6-951b169b7889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.074020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bb21d-4ddb-4a6c-9dc6-951b169b7889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:34.075960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bb21d-4ddb-4a6c-9dc6-951b169b7889 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:34.091276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 934/1867] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:34.093003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7162d8-598e-46e2-8ff9-237c582b2d2e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.093380 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7162d8-598e-46e2-8ff9-237c582b2d2e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.093763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7162d8-598e-46e2-8ff9-237c582b2d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.115901 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-6b7162d8-598e-46e2-8ff9-237c582b2d2e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:34.117497 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6b7162d8-598e-46e2-8ff9-237c582b2d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:34.127789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 935/1868] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:34.128687 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7162d8-598e-46e2-8ff9-237c582b2d2e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:34.142369 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7162d8-598e-46e2-8ff9-237c582b2d2e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:34.150934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33593836-8ae3-4c5c-95bf-59e05b044dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:34.151861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33593836-8ae3-4c5c-95bf-59e05b044dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.152023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33593836-8ae3-4c5c-95bf-59e05b044dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.153154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33593836-8ae3-4c5c-95bf-59e05b044dc2 admin admin] PATH_INFO: `/v3/projects/fbbe037cacbb4a0d8925699eb6b5df2c/users/f2f9e01cf62047a4b32671960e0d7677/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.155496 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33593836-8ae3-4c5c-95bf-59e05b044dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbbe037cacbb4a0d8925699eb6b5df2c, user_id=f2f9e01cf62047a4b32671960e0d7677, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:34.159975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33593836-8ae3-4c5c-95bf-59e05b044dc2 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:34.163728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ca9ae0d6-aae3-471c-9513-b1ffd4f7df81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '619e3cb8198c460db8bf8292fb123597'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:34.164170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ca9ae0d6-aae3-471c-9513-b1ffd4f7df81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '619e3cb8198c460db8bf8292fb123597'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:34.164830 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ca9ae0d6-aae3-471c-9513-b1ffd4f7df81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '619e3cb8198c460db8bf8292fb123597'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:34.176841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 936/1869] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:34 2022] PUT /identity/v3/projects/fbbe037cacbb4a0d8925699eb6b5df2c/users/f2f9e01cf62047a4b32671960e0d7677/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:34.177821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33593836-8ae3-4c5c-95bf-59e05b044dc2 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:34.180680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-33593836-8ae3-4c5c-95bf-59e05b044dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:34.183712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ca9ae0d6-aae3-471c-9513-b1ffd4f7df81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '619e3cb8198c460db8bf8292fb123597'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:34.191447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ca9ae0d6-aae3-471c-9513-b1ffd4f7df81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '619e3cb8198c460db8bf8292fb123597'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:34.209351 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 934/1870] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:33 2022] DELETE /identity/v3/users/619e3cb8198c460db8bf8292fb123597 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:34.211280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ae881-3672-4cad-afbd-9cbc63096750 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.211527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ae881-3672-4cad-afbd-9cbc63096750 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.211937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ae881-3672-4cad-afbd-9cbc63096750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.257617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ef9ae881-3672-4cad-afbd-9cbc63096750 None None] MFA Rules not processed for user `f2f9e01cf62047a4b32671960e0d7677`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:34.258839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ef9ae881-3672-4cad-afbd-9cbc63096750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:34.273213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33593836-8ae3-4c5c-95bf-59e05b044dc2 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:34.346554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:34.363029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459aed0-b4f8-4ae2-90a8-d456aba7b17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:34.363651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459aed0-b4f8-4ae2-90a8-d456aba7b17f admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.363934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459aed0-b4f8-4ae2-90a8-d456aba7b17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.364207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459aed0-b4f8-4ae2-90a8-d456aba7b17f admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.364881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459aed0-b4f8-4ae2-90a8-d456aba7b17f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:34.365185 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459aed0-b4f8-4ae2-90a8-d456aba7b17f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:34.366783 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459aed0-b4f8-4ae2-90a8-d456aba7b17f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:34.378468 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 937/1871] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:32:34.379815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d5d4-380f-4a4d-9e90-49d2aa5e3eb1 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.380225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d5d4-380f-4a4d-9e90-49d2aa5e3eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.380504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d5d4-380f-4a4d-9e90-49d2aa5e3eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.384862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 935/1872] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:34.385680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ae881-3672-4cad-afbd-9cbc63096750 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:34.397066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ae881-3672-4cad-afbd-9cbc63096750 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:34.402443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385ed2b-c7f5-477e-a560-64fecd4bc891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:34.402983 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385ed2b-c7f5-477e-a560-64fecd4bc891 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.403312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385ed2b-c7f5-477e-a560-64fecd4bc891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.403523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385ed2b-c7f5-477e-a560-64fecd4bc891 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.404149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385ed2b-c7f5-477e-a560-64fecd4bc891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:34.404773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-cd55d5d4-380f-4a4d-9e90-49d2aa5e3eb1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:34.406033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-cd55d5d4-380f-4a4d-9e90-49d2aa5e3eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:34.406123 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385ed2b-c7f5-477e-a560-64fecd4bc891 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:34.414111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 938/1873] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:34.417716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d5d4-380f-4a4d-9e90-49d2aa5e3eb1 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:34.419259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-cd55d5d4-380f-4a4d-9e90-49d2aa5e3eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:34.443185 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 936/1874] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:34 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:34.443185 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385ed2b-c7f5-477e-a560-64fecd4bc891 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:34.446120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d385ed2b-c7f5-477e-a560-64fecd4bc891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:34.489114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d5d4-380f-4a4d-9e90-49d2aa5e3eb1 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:34.502130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:34.518767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385ed2b-c7f5-477e-a560-64fecd4bc891 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:34.524706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead5ef6-d0a7-42e0-918c-675245e210dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:34.526347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead5ef6-d0a7-42e0-918c-675245e210dc service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.527124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead5ef6-d0a7-42e0-918c-675245e210dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.527124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead5ef6-d0a7-42e0-918c-675245e210dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.549816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e132caf-513b-4ec1-b2f1-845dcc23d955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:34.550564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e132caf-513b-4ec1-b2f1-845dcc23d955 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.551137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e132caf-513b-4ec1-b2f1-845dcc23d955 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.551300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e132caf-513b-4ec1-b2f1-845dcc23d955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.594654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead5ef6-d0a7-42e0-918c-675245e210dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:34.596091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead5ef6-d0a7-42e0-918c-675245e210dc service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:34.605648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 937/1875] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:34.606434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead5ef6-d0a7-42e0-918c-675245e210dc service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:34.616178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead5ef6-d0a7-42e0-918c-675245e210dc service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:34.617159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e132caf-513b-4ec1-b2f1-845dcc23d955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:34.618769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e132caf-513b-4ec1-b2f1-845dcc23d955 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:34.621454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a92eca-b61d-4189-b104-794c0f4f5260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:34.622065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a92eca-b61d-4189-b104-794c0f4f5260 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.622343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a92eca-b61d-4189-b104-794c0f4f5260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.622613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a92eca-b61d-4189-b104-794c0f4f5260 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.623230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a92eca-b61d-4189-b104-794c0f4f5260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:34.625183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a92eca-b61d-4189-b104-794c0f4f5260 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:34.630129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 939/1876] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:34.668798 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 938/1877] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:34 2022] POST /identity/v3/users => generated 285 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:34.673662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e132caf-513b-4ec1-b2f1-845dcc23d955 service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:34.692090 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e132caf-513b-4ec1-b2f1-845dcc23d955 service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:34.708619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be66c72-a79a-4a7d-ac8e-1f0d4455d66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:34.709574 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be66c72-a79a-4a7d-ac8e-1f0d4455d66d admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.710199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be66c72-a79a-4a7d-ac8e-1f0d4455d66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.710592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be66c72-a79a-4a7d-ac8e-1f0d4455d66d admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.711500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be66c72-a79a-4a7d-ac8e-1f0d4455d66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:34.714672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be66c72-a79a-4a7d-ac8e-1f0d4455d66d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:34.732800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 940/1878] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:34.736091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a92eca-b61d-4189-b104-794c0f4f5260 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:34.746996 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a92eca-b61d-4189-b104-794c0f4f5260 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:34.751908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1ca34-031a-488f-9b1c-a7afe2646ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:34.752599 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1ca34-031a-488f-9b1c-a7afe2646ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.752972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1ca34-031a-488f-9b1c-a7afe2646ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.753243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1ca34-031a-488f-9b1c-a7afe2646ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.753958 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1ca34-031a-488f-9b1c-a7afe2646ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:34.755705 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1ca34-031a-488f-9b1c-a7afe2646ae2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:34.763411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be66c72-a79a-4a7d-ac8e-1f0d4455d66d admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:34.773150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 939/1879] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:34.774246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1ca34-031a-488f-9b1c-a7afe2646ae2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:34.783663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be66c72-a79a-4a7d-ac8e-1f0d4455d66d admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:34.785189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1ca34-031a-488f-9b1c-a7afe2646ae2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:34.788540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d6873-ee13-4e4d-a7e1-646639f01e71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:34.789340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d6873-ee13-4e4d-a7e1-646639f01e71 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.789712 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d6873-ee13-4e4d-a7e1-646639f01e71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.790021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d6873-ee13-4e4d-a7e1-646639f01e71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.790239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65c522-4053-43cf-a896-1278274b6369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:34.790999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65c522-4053-43cf-a896-1278274b6369 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.791353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65c522-4053-43cf-a896-1278274b6369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.791708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65c522-4053-43cf-a896-1278274b6369 admin admin] PATH_INFO: `/v3/projects/559646cea52a407e9d71ea5475fa3319/users/d1547dbf6d024e4db86d6a79771f6ad7/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.792282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-429d6873-ee13-4e4d-a7e1-646639f01e71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:34.809138 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65c522-4053-43cf-a896-1278274b6369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=559646cea52a407e9d71ea5475fa3319, user_id=d1547dbf6d024e4db86d6a79771f6ad7, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:34.811928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65c522-4053-43cf-a896-1278274b6369 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:34.827995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:34.836956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 940/1880] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:34 2022] PUT /identity/v3/projects/559646cea52a407e9d71ea5475fa3319/users/d1547dbf6d024e4db86d6a79771f6ad7/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:34.842634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1952aa-47fc-4f2f-b11d-44dba5672cf3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:34.842923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1952aa-47fc-4f2f-b11d-44dba5672cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:34.843200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1952aa-47fc-4f2f-b11d-44dba5672cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:34.867869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d6873-ee13-4e4d-a7e1-646639f01e71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:34.870082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d6873-ee13-4e4d-a7e1-646639f01e71 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:34.877722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-eb1952aa-47fc-4f2f-b11d-44dba5672cf3 None None] MFA Rules not processed for user `d1547dbf6d024e4db86d6a79771f6ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:34.879053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-eb1952aa-47fc-4f2f-b11d-44dba5672cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:34.881208 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 941/1881] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:34.965825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 941/1882] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:34.968350 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d6873-ee13-4e4d-a7e1-646639f01e71 service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:35.000958 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d6873-ee13-4e4d-a7e1-646639f01e71 service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:35.091868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b1f6e-8f66-4781-bf33-4381e21495cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:35.094172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b1f6e-8f66-4781-bf33-4381e21495cd admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:35.094172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b1f6e-8f66-4781-bf33-4381e21495cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:35.094172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b1f6e-8f66-4781-bf33-4381e21495cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:35.094966 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b1f6e-8f66-4781-bf33-4381e21495cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:35.097451 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b1f6e-8f66-4781-bf33-4381e21495cd admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:35.153323 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1952aa-47fc-4f2f-b11d-44dba5672cf3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:35.154898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-eb1952aa-47fc-4f2f-b11d-44dba5672cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:35.163006 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 942/1883] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:34 2022] POST /identity/v3/projects => generated 369 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:35.164663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b1f6e-8f66-4781-bf33-4381e21495cd admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:35.185140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1952aa-47fc-4f2f-b11d-44dba5672cf3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:35.223932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:35.271355 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:35.272147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:35.272460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:35.273067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] PATH_INFO: `/v3/projects/9440563c215d40a7b7371c784c2449af` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:35.276232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9440563c215d40a7b7371c784c2449af)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:35.287020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:35.370800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b1f6e-8f66-4781-bf33-4381e21495cd admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:35.382925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534e7e8-576b-45b2-bc62-b940599350af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:35.383681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e7e8-576b-45b2-bc62-b940599350af admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:35.384223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e7e8-576b-45b2-bc62-b940599350af admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:35.386110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e7e8-576b-45b2-bc62-b940599350af admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:35.387174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534e7e8-576b-45b2-bc62-b940599350af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:35.391767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534e7e8-576b-45b2-bc62-b940599350af admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:35.419945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 943/1884] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:35 2022] POST /identity/v3/users => generated 291 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:35.425149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534e7e8-576b-45b2-bc62-b940599350af admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:35.426119 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9440563c215d40a7b7371c784c2449af', 'request_id': 'req-12948e09-840c-4793-b523-eb0a7e4daafa'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:35.439059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534e7e8-576b-45b2-bc62-b940599350af admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:35.439340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9440563c215d40a7b7371c784c2449af', 'request_id': 'req-12948e09-840c-4793-b523-eb0a7e4daafa'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:35.450540 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9440563c215d40a7b7371c784c2449af', 'request_id': 'req-12948e09-840c-4793-b523-eb0a7e4daafa'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:35.485979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] The token cache is being invalidate because project 9440563c215d40a7b7371c784c2449af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:35.486524 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:35.487940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:35.489740 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 942/1885] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:35 2022] DELETE /identity/v3/projects/9440563c215d40a7b7371c784c2449af => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:35.492399 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:35.494873 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:35.496390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:35.520332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349a94f-9eb0-48a2-90d7-77d33248ae1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:35.521590 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349a94f-9eb0-48a2-90d7-77d33248ae1a admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:35.522150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349a94f-9eb0-48a2-90d7-77d33248ae1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:35.522742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349a94f-9eb0-48a2-90d7-77d33248ae1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:35.523577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349a94f-9eb0-48a2-90d7-77d33248ae1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:35.526692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349a94f-9eb0-48a2-90d7-77d33248ae1a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:35.543063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 944/1886] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:35.544072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349a94f-9eb0-48a2-90d7-77d33248ae1a admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:35.546340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-4349a94f-9eb0-48a2-90d7-77d33248ae1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:35.563846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349a94f-9eb0-48a2-90d7-77d33248ae1a admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:35.575292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aae78f-f898-46af-a42d-c6096371c38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:35.576182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aae78f-f898-46af-a42d-c6096371c38e admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:35.576444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aae78f-f898-46af-a42d-c6096371c38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:35.576912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aae78f-f898-46af-a42d-c6096371c38e admin admin] PATH_INFO: `/v3/projects/f48faf8c8da44ff08d7ec25fcc66b11d/users/4f68d8ca6eb44572af56009b360f9c34/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:35.606505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12948e09-840c-4793-b523-eb0a7e4daafa admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:35.608861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aae78f-f898-46af-a42d-c6096371c38e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48faf8c8da44ff08d7ec25fcc66b11d, user_id=4f68d8ca6eb44572af56009b360f9c34, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:35.612040 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aae78f-f898-46af-a42d-c6096371c38e admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:35.625441 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa84ce9-f9ae-4834-b382-4ca0c49d2575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:35.626276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa84ce9-f9ae-4834-b382-4ca0c49d2575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:35.626739 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa84ce9-f9ae-4834-b382-4ca0c49d2575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:35.627122 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa84ce9-f9ae-4834-b382-4ca0c49d2575 admin admin] PATH_INFO: `/v3/users/9cc723d89c1d4b1e92e619fca0b9c1cf` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:35.628549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa84ce9-f9ae-4834-b382-4ca0c49d2575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cc723d89c1d4b1e92e619fca0b9c1cf)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:35.630663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa84ce9-f9ae-4834-b382-4ca0c49d2575 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:35.658586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 945/1887] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:35 2022] PUT /identity/v3/projects/f48faf8c8da44ff08d7ec25fcc66b11d/users/4f68d8ca6eb44572af56009b360f9c34/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:35.659940 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aae78f-f898-46af-a42d-c6096371c38e admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:35.731436 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-eaa84ce9-f9ae-4834-b382-4ca0c49d2575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cc723d89c1d4b1e92e619fca0b9c1cf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:35.733484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-eaa84ce9-f9ae-4834-b382-4ca0c49d2575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cc723d89c1d4b1e92e619fca0b9c1cf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:35.736383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-eaa84ce9-f9ae-4834-b382-4ca0c49d2575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cc723d89c1d4b1e92e619fca0b9c1cf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:35.746478 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:35.749514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-eaa84ce9-f9ae-4834-b382-4ca0c49d2575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cc723d89c1d4b1e92e619fca0b9c1cf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:35.753382 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aae78f-f898-46af-a42d-c6096371c38e admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:35.761064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384cc09-87ef-43e6-9828-4431a7575b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:35.761779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384cc09-87ef-43e6-9828-4431a7575b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:35.762062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384cc09-87ef-43e6-9828-4431a7575b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:35.762301 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384cc09-87ef-43e6-9828-4431a7575b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:35.762725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-eaa84ce9-f9ae-4834-b382-4ca0c49d2575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cc723d89c1d4b1e92e619fca0b9c1cf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:35.762863 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384cc09-87ef-43e6-9828-4431a7575b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:35.764475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384cc09-87ef-43e6-9828-4431a7575b04 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:35.779506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 946/1888] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:35.781003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384cc09-87ef-43e6-9828-4431a7575b04 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:35.782725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 943/1889] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:35 2022] DELETE /identity/v3/users/9cc723d89c1d4b1e92e619fca0b9c1cf => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:35.782846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3384cc09-87ef-43e6-9828-4431a7575b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:35.797426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384cc09-87ef-43e6-9828-4431a7575b04 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:35.870718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5204d-6adc-44a6-9299-93ab430369fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:35.871568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5204d-6adc-44a6-9299-93ab430369fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:35.871838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5204d-6adc-44a6-9299-93ab430369fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:35.872158 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5204d-6adc-44a6-9299-93ab430369fd admin admin] PATH_INFO: `/v3/projects/f48faf8c8da44ff08d7ec25fcc66b11d/users/4f68d8ca6eb44572af56009b360f9c34/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:35.874223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5204d-6adc-44a6-9299-93ab430369fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48faf8c8da44ff08d7ec25fcc66b11d, user_id=4f68d8ca6eb44572af56009b360f9c34, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:35.876936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5204d-6adc-44a6-9299-93ab430369fd admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:35.888890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 947/1890] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:35 2022] PUT /identity/v3/projects/f48faf8c8da44ff08d7ec25fcc66b11d/users/4f68d8ca6eb44572af56009b360f9c34/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:35.899034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79bd0e-44c8-4e3d-8075-fce9778d0c73 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:35.899330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79bd0e-44c8-4e3d-8075-fce9778d0c73 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:35.899611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79bd0e-44c8-4e3d-8075-fce9778d0c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:35.933045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4c79bd0e-44c8-4e3d-8075-fce9778d0c73 None None] MFA Rules not processed for user `4f68d8ca6eb44572af56009b360f9c34`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:35.934335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4c79bd0e-44c8-4e3d-8075-fce9778d0c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:36.011970 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 944/1891] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:36.037077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5204d-6adc-44a6-9299-93ab430369fd admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:36.038846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d4b5204d-6adc-44a6-9299-93ab430369fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:36.092255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79bd0e-44c8-4e3d-8075-fce9778d0c73 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:36.093783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4c79bd0e-44c8-4e3d-8075-fce9778d0c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:36.100787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:36.109567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5204d-6adc-44a6-9299-93ab430369fd admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:36.113519 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736cef2-5286-401c-b276-4e86ba0e30d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:36.114172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736cef2-5286-401c-b276-4e86ba0e30d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:36.114456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736cef2-5286-401c-b276-4e86ba0e30d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:36.114729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736cef2-5286-401c-b276-4e86ba0e30d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:36.126639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736cef2-5286-401c-b276-4e86ba0e30d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:36.128151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736cef2-5286-401c-b276-4e86ba0e30d5 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:36.139529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 948/1892] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:36.177858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79bd0e-44c8-4e3d-8075-fce9778d0c73 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:36.182251 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:36.182894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:36.183171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:36.183442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] PATH_INFO: `/v3/projects/6654a795f2c747e68bd6f599e0819071` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:36.184903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6654a795f2c747e68bd6f599e0819071)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:36.185433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736cef2-5286-401c-b276-4e86ba0e30d5 service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:36.186810 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:36.186927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-4736cef2-5286-401c-b276-4e86ba0e30d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:36.198470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736cef2-5286-401c-b276-4e86ba0e30d5 service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:36.202246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264b005-b42e-4cb7-b4b1-21776e7a0f37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:36.202840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264b005-b42e-4cb7-b4b1-21776e7a0f37 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:36.203120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264b005-b42e-4cb7-b4b1-21776e7a0f37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:36.203392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264b005-b42e-4cb7-b4b1-21776e7a0f37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:36.205123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0264b005-b42e-4cb7-b4b1-21776e7a0f37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:36.235816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6654a795f2c747e68bd6f599e0819071', 'request_id': 'req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:36.241833 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6654a795f2c747e68bd6f599e0819071', 'request_id': 'req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:36.248720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6654a795f2c747e68bd6f599e0819071', 'request_id': 'req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:36.251720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:36.265807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] The token cache is being invalidate because project 6654a795f2c747e68bd6f599e0819071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:36.266106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:36.266343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264b005-b42e-4cb7-b4b1-21776e7a0f37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:36.266761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:36.267772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264b005-b42e-4cb7-b4b1-21776e7a0f37 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:36.268542 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 945/1893] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:36 2022] DELETE /identity/v3/projects/6654a795f2c747e68bd6f599e0819071 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:36.272334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:36.274309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:36.277715 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 949/1894] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:36.340360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e5f19-de2d-4ebc-a95e-e14d905ae2a2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:36.345072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064135ac-d45d-4ee8-8b08-4dd0a63076c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:36.345771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064135ac-d45d-4ee8-8b08-4dd0a63076c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:36.346071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064135ac-d45d-4ee8-8b08-4dd0a63076c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:36.346349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064135ac-d45d-4ee8-8b08-4dd0a63076c2 admin admin] PATH_INFO: `/v3/users/008d17d7079e43b7887cfc8792a3f4a2` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:36.347523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064135ac-d45d-4ee8-8b08-4dd0a63076c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=008d17d7079e43b7887cfc8792a3f4a2)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:36.349603 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064135ac-d45d-4ee8-8b08-4dd0a63076c2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:36.425822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-064135ac-d45d-4ee8-8b08-4dd0a63076c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '008d17d7079e43b7887cfc8792a3f4a2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:36.427407 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-064135ac-d45d-4ee8-8b08-4dd0a63076c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '008d17d7079e43b7887cfc8792a3f4a2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:36.432899 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-064135ac-d45d-4ee8-8b08-4dd0a63076c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '008d17d7079e43b7887cfc8792a3f4a2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:36.458086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-064135ac-d45d-4ee8-8b08-4dd0a63076c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '008d17d7079e43b7887cfc8792a3f4a2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:36.467252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-064135ac-d45d-4ee8-8b08-4dd0a63076c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '008d17d7079e43b7887cfc8792a3f4a2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:36.520448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 946/1895] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:36 2022] DELETE /identity/v3/users/008d17d7079e43b7887cfc8792a3f4a2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:38.288987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264b005-b42e-4cb7-b4b1-21776e7a0f37 service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:38.318183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264b005-b42e-4cb7-b4b1-21776e7a0f37 service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:38.534545 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6158d3-5788-459c-bdc4-5335ebbcfa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:38.535198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6158d3-5788-459c-bdc4-5335ebbcfa26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:38.535479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6158d3-5788-459c-bdc4-5335ebbcfa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:38.535754 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6158d3-5788-459c-bdc4-5335ebbcfa26 admin admin] PATH_INFO: `/v3/projects/1c3f443988c1407b8032a8a73e6a16a7` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:38.536946 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6158d3-5788-459c-bdc4-5335ebbcfa26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3f443988c1407b8032a8a73e6a16a7)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:38.538867 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6158d3-5788-459c-bdc4-5335ebbcfa26 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:38.644252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cf6158d3-5788-459c-bdc4-5335ebbcfa26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3f443988c1407b8032a8a73e6a16a7', 'request_id': 'req-cf6158d3-5788-459c-bdc4-5335ebbcfa26'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:38.652816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cf6158d3-5788-459c-bdc4-5335ebbcfa26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3f443988c1407b8032a8a73e6a16a7', 'request_id': 'req-cf6158d3-5788-459c-bdc4-5335ebbcfa26'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:38.663360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cf6158d3-5788-459c-bdc4-5335ebbcfa26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3f443988c1407b8032a8a73e6a16a7', 'request_id': 'req-cf6158d3-5788-459c-bdc4-5335ebbcfa26'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:38.696707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cf6158d3-5788-459c-bdc4-5335ebbcfa26 admin admin] The token cache is being invalidate because project 1c3f443988c1407b8032a8a73e6a16a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:38.697687 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cf6158d3-5788-459c-bdc4-5335ebbcfa26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:38.698427 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cf6158d3-5788-459c-bdc4-5335ebbcfa26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:38.703943 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 950/1896] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:38 2022] DELETE /identity/v3/projects/1c3f443988c1407b8032a8a73e6a16a7 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:38.705841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f5edf-dd90-435a-be5b-e766170a9c76 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:38.706310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f5edf-dd90-435a-be5b-e766170a9c76 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:38.706473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f5edf-dd90-435a-be5b-e766170a9c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:38.736616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-099f5edf-dd90-435a-be5b-e766170a9c76 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:38.738062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-099f5edf-dd90-435a-be5b-e766170a9c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:38.822205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 947/1897] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:38.830312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c405a-d645-4347-98c3-f6d1cbdf5f6b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:38.830910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c405a-d645-4347-98c3-f6d1cbdf5f6b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:38.831287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c405a-d645-4347-98c3-f6d1cbdf5f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:38.864165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-876c405a-d645-4347-98c3-f6d1cbdf5f6b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:38.865563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-876c405a-d645-4347-98c3-f6d1cbdf5f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:38.878276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 951/1898] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:38.881772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f5edf-dd90-435a-be5b-e766170a9c76 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:38.891197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f5edf-dd90-435a-be5b-e766170a9c76 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:38.898465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d44f2-ba79-4e21-a668-2e433359460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:38.899161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d44f2-ba79-4e21-a668-2e433359460f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:38.899514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d44f2-ba79-4e21-a668-2e433359460f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:38.899891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d44f2-ba79-4e21-a668-2e433359460f admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:38.901303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d44f2-ba79-4e21-a668-2e433359460f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:38.901693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d44f2-ba79-4e21-a668-2e433359460f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:38.903376 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d44f2-ba79-4e21-a668-2e433359460f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:38.912961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:38.920639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 948/1899] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:32:38.921958 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c405a-d645-4347-98c3-f6d1cbdf5f6b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:38.932786 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c405a-d645-4347-98c3-f6d1cbdf5f6b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:38.937688 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1426784-9976-4335-b494-bc44466dc328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:38.938297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1426784-9976-4335-b494-bc44466dc328 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:38.938624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1426784-9976-4335-b494-bc44466dc328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:38.939033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1426784-9976-4335-b494-bc44466dc328 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:38.939696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1426784-9976-4335-b494-bc44466dc328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:38.941677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1426784-9976-4335-b494-bc44466dc328 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:38.952216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:38.976658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 952/1900] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:38 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:38.982117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d44f2-ba79-4e21-a668-2e433359460f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:38.994690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d44f2-ba79-4e21-a668-2e433359460f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:39.067860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac1898-6dd0-4eea-bef1-85859b0f6c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:39.068513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac1898-6dd0-4eea-bef1-85859b0f6c41 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:39.068993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac1898-6dd0-4eea-bef1-85859b0f6c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:39.069277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac1898-6dd0-4eea-bef1-85859b0f6c41 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:39.069915 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac1898-6dd0-4eea-bef1-85859b0f6c41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:39.072438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac1898-6dd0-4eea-bef1-85859b0f6c41 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:39.096430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 949/1901] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:38 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:32:39.100601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1426784-9976-4335-b494-bc44466dc328 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:39.111547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1426784-9976-4335-b494-bc44466dc328 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:39.118062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92153be-cf51-457f-88f8-86b4c96d1054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:39.118784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92153be-cf51-457f-88f8-86b4c96d1054 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:39.119150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92153be-cf51-457f-88f8-86b4c96d1054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:39.119533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92153be-cf51-457f-88f8-86b4c96d1054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:39.120209 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92153be-cf51-457f-88f8-86b4c96d1054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:39.121848 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92153be-cf51-457f-88f8-86b4c96d1054 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:39.138033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 953/1902] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:39.139140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac1898-6dd0-4eea-bef1-85859b0f6c41 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:39.153500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac1898-6dd0-4eea-bef1-85859b0f6c41 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:39.159142 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c249de6-5ee6-45d4-b806-f08ce3255826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:39.159925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c249de6-5ee6-45d4-b806-f08ce3255826 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:39.160342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c249de6-5ee6-45d4-b806-f08ce3255826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:39.160797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c249de6-5ee6-45d4-b806-f08ce3255826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:39.173699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c249de6-5ee6-45d4-b806-f08ce3255826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:39.173699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c249de6-5ee6-45d4-b806-f08ce3255826 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:39.177479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 950/1903] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:39.182015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92153be-cf51-457f-88f8-86b4c96d1054 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:39.193071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92153be-cf51-457f-88f8-86b4c96d1054 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:39.198334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045df838-6312-4281-8106-764a7583c2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:39.199002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045df838-6312-4281-8106-764a7583c2d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:39.199285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045df838-6312-4281-8106-764a7583c2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:39.199560 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045df838-6312-4281-8106-764a7583c2d9 admin admin] PATH_INFO: `/v3/projects/a549f3d5fb9443409faa06cf7b00d8ca/users/7536a92033384383bd625b6c969ebd38/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:39.216445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045df838-6312-4281-8106-764a7583c2d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a549f3d5fb9443409faa06cf7b00d8ca, user_id=7536a92033384383bd625b6c969ebd38, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:39.219719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045df838-6312-4281-8106-764a7583c2d9 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:39.233804 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 954/1904] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:39 2022] PUT /identity/v3/projects/a549f3d5fb9443409faa06cf7b00d8ca/users/7536a92033384383bd625b6c969ebd38/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:39.239177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf241fc-78e6-4056-b5c2-2b7a7f475be2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:39.239566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf241fc-78e6-4056-b5c2-2b7a7f475be2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:39.239906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf241fc-78e6-4056-b5c2-2b7a7f475be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:39.258929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1bf241fc-78e6-4056-b5c2-2b7a7f475be2 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:39.260791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1bf241fc-78e6-4056-b5c2-2b7a7f475be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:39.329275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 951/1905] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:39.339466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045df838-6312-4281-8106-764a7583c2d9 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:39.341318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-045df838-6312-4281-8106-764a7583c2d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:39.404983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045df838-6312-4281-8106-764a7583c2d9 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:39.408742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869a752-6e95-4dfa-85d8-9f4900e1ee97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:39.409406 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869a752-6e95-4dfa-85d8-9f4900e1ee97 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:39.409737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869a752-6e95-4dfa-85d8-9f4900e1ee97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:39.409958 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869a752-6e95-4dfa-85d8-9f4900e1ee97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:39.420175 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869a752-6e95-4dfa-85d8-9f4900e1ee97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:39.421893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869a752-6e95-4dfa-85d8-9f4900e1ee97 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:39.435788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 955/1906] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:40.270184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf241fc-78e6-4056-b5c2-2b7a7f475be2 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:40.272991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1bf241fc-78e6-4056-b5c2-2b7a7f475be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:40.293706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf241fc-78e6-4056-b5c2-2b7a7f475be2 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:40.301931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a387339-c7b1-4820-8b2d-65689c0e3060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:40.302746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a387339-c7b1-4820-8b2d-65689c0e3060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:40.303312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a387339-c7b1-4820-8b2d-65689c0e3060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:40.303778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a387339-c7b1-4820-8b2d-65689c0e3060 admin admin] PATH_INFO: `/v3/users/d1547dbf6d024e4db86d6a79771f6ad7` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:40.306565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a387339-c7b1-4820-8b2d-65689c0e3060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1547dbf6d024e4db86d6a79771f6ad7)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:40.309951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a387339-c7b1-4820-8b2d-65689c0e3060 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:40.446167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2a387339-c7b1-4820-8b2d-65689c0e3060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1547dbf6d024e4db86d6a79771f6ad7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:40.446630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869a752-6e95-4dfa-85d8-9f4900e1ee97 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:40.448004 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2a387339-c7b1-4820-8b2d-65689c0e3060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1547dbf6d024e4db86d6a79771f6ad7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:40.450342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2a387339-c7b1-4820-8b2d-65689c0e3060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1547dbf6d024e4db86d6a79771f6ad7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:40.456462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b869a752-6e95-4dfa-85d8-9f4900e1ee97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:40.467583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2a387339-c7b1-4820-8b2d-65689c0e3060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1547dbf6d024e4db86d6a79771f6ad7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:40.473730 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869a752-6e95-4dfa-85d8-9f4900e1ee97 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:40.477070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2a387339-c7b1-4820-8b2d-65689c0e3060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1547dbf6d024e4db86d6a79771f6ad7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:40.479759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:40.481068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:40.482547 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:40.483532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:40.484046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] PATH_INFO: `/v3/users/5d56eeaf459640b6b81cab1a27c9cffe` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:40.486939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d56eeaf459640b6b81cab1a27c9cffe)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:40.490344 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:40.505770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 952/1907] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:40 2022] DELETE /identity/v3/users/d1547dbf6d024e4db86d6a79771f6ad7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:40.512236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f32ad-20f7-4804-a49a-93f58a022fc7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:40.513000 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f32ad-20f7-4804-a49a-93f58a022fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:40.513625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f32ad-20f7-4804-a49a-93f58a022fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:40.553850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a45f32ad-20f7-4804-a49a-93f58a022fc7 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:40.556934 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a45f32ad-20f7-4804-a49a-93f58a022fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:40.581584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d56eeaf459640b6b81cab1a27c9cffe'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:40.582282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d56eeaf459640b6b81cab1a27c9cffe'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:40.582904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d56eeaf459640b6b81cab1a27c9cffe'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:40.595506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d56eeaf459640b6b81cab1a27c9cffe'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:40.614680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d56eeaf459640b6b81cab1a27c9cffe'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:40.641501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 953/1909] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:40.642357 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 956/1909] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:40 2022] DELETE /identity/v3/users/5d56eeaf459640b6b81cab1a27c9cffe => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:40.648781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a5bac-ae37-4327-8b34-2b1dd305c0d1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:40.649125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a5bac-ae37-4327-8b34-2b1dd305c0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:40.649451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a5bac-ae37-4327-8b34-2b1dd305c0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:40.672614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-813a5bac-ae37-4327-8b34-2b1dd305c0d1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:40.674090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-813a5bac-ae37-4327-8b34-2b1dd305c0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:40.686517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:40.688527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:40.746569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:40.767555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 954/1910] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:40.785852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a5bac-ae37-4327-8b34-2b1dd305c0d1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:40.788328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-813a5bac-ae37-4327-8b34-2b1dd305c0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:40.792132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d861f4-d7e7-4159-9d84-463375af7188 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:40.796432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d6aee-86f3-49c8-a56e-27b3226e46a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:40.797090 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d6aee-86f3-49c8-a56e-27b3226e46a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:40.797713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d6aee-86f3-49c8-a56e-27b3226e46a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:40.798428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d6aee-86f3-49c8-a56e-27b3226e46a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:40.801729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a5bac-ae37-4327-8b34-2b1dd305c0d1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:40.807863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289362b2-aac0-4a59-9863-bbde900f2386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:40.808547 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289362b2-aac0-4a59-9863-bbde900f2386 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:40.808815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289362b2-aac0-4a59-9863-bbde900f2386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:40.809249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289362b2-aac0-4a59-9863-bbde900f2386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:40.810418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d6aee-86f3-49c8-a56e-27b3226e46a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:40.811984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d6aee-86f3-49c8-a56e-27b3226e46a7 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:40.821200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289362b2-aac0-4a59-9863-bbde900f2386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:40.823876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289362b2-aac0-4a59-9863-bbde900f2386 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:40.836669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 957/1911] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:40.839883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 955/1912] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:41.199380 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d6aee-86f3-49c8-a56e-27b3226e46a7 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:41.201012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f85d6aee-86f3-49c8-a56e-27b3226e46a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:41.216164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d6aee-86f3-49c8-a56e-27b3226e46a7 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:41.221789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae3a03-3d63-4cd8-a901-571604b6a9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:41.222664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae3a03-3d63-4cd8-a901-571604b6a9df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:41.223161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae3a03-3d63-4cd8-a901-571604b6a9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:41.223631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae3a03-3d63-4cd8-a901-571604b6a9df admin admin] PATH_INFO: `/v3/projects/eff36f36cf69438ea25483a680ba0a28` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:41.225197 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae3a03-3d63-4cd8-a901-571604b6a9df admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff36f36cf69438ea25483a680ba0a28)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:41.227503 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae3a03-3d63-4cd8-a901-571604b6a9df admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:41.281870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-40ae3a03-3d63-4cd8-a901-571604b6a9df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff36f36cf69438ea25483a680ba0a28', 'request_id': 'req-40ae3a03-3d63-4cd8-a901-571604b6a9df'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:41.290153 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-40ae3a03-3d63-4cd8-a901-571604b6a9df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff36f36cf69438ea25483a680ba0a28', 'request_id': 'req-40ae3a03-3d63-4cd8-a901-571604b6a9df'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:41.298328 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-40ae3a03-3d63-4cd8-a901-571604b6a9df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff36f36cf69438ea25483a680ba0a28', 'request_id': 'req-40ae3a03-3d63-4cd8-a901-571604b6a9df'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:41.315046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-40ae3a03-3d63-4cd8-a901-571604b6a9df admin admin] The token cache is being invalidate because project eff36f36cf69438ea25483a680ba0a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:41.315357 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-40ae3a03-3d63-4cd8-a901-571604b6a9df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:41.315978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-40ae3a03-3d63-4cd8-a901-571604b6a9df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:41.317803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 958/1913] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:41 2022] DELETE /identity/v3/projects/eff36f36cf69438ea25483a680ba0a28 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:41.321733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289362b2-aac0-4a59-9863-bbde900f2386 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:41.334115 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289362b2-aac0-4a59-9863-bbde900f2386 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:41.430219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:41.431879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:41.432326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:41.432802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] PATH_INFO: `/v3/users/f2f9e01cf62047a4b32671960e0d7677` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:41.434178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2f9e01cf62047a4b32671960e0d7677)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:41.436428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:41.480274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:INSERT=1,keystone:DELETE=4,keystone:SELECT=95 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:41.520087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2f9e01cf62047a4b32671960e0d7677'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:41.521170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2f9e01cf62047a4b32671960e0d7677'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:41.521860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb37cab-283b-4ed7-bab3-8368ef50f146 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:41.522321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2f9e01cf62047a4b32671960e0d7677'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:41.522412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb37cab-283b-4ed7-bab3-8368ef50f146 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:41.522412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb37cab-283b-4ed7-bab3-8368ef50f146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:41.537706 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2f9e01cf62047a4b32671960e0d7677'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:41.552500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2f9e01cf62047a4b32671960e0d7677'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:41.562345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-bcb37cab-283b-4ed7-bab3-8368ef50f146 None None] MFA Rules not processed for user `7536a92033384383bd625b6c969ebd38`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:41.564105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-bcb37cab-283b-4ed7-bab3-8368ef50f146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:41.588926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 956/1914] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:41 2022] DELETE /identity/v3/users/f2f9e01cf62047a4b32671960e0d7677 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:41.697882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 959/1915] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:41.709273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:41.711299 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:41.780444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8da6e3-fc8f-4efa-8f96-32823007c75b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:41.784771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26310a-268c-4563-bf23-e980965c83b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:41.785390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26310a-268c-4563-bf23-e980965c83b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:41.785677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26310a-268c-4563-bf23-e980965c83b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:41.786316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26310a-268c-4563-bf23-e980965c83b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:41.837887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb37cab-283b-4ed7-bab3-8368ef50f146 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:41.839695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-bcb37cab-283b-4ed7-bab3-8368ef50f146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:41.861769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb37cab-283b-4ed7-bab3-8368ef50f146 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:41.884826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26310a-268c-4563-bf23-e980965c83b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:41.887037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26310a-268c-4563-bf23-e980965c83b4 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:41.906728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 957/1916] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:41.935378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:41.965505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:41.966191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:41.966479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:41.966755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] PATH_INFO: `/v3/projects/559646cea52a407e9d71ea5475fa3319` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:41.968429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=559646cea52a407e9d71ea5475fa3319)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:41.973053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:42.064314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '559646cea52a407e9d71ea5475fa3319', 'request_id': 'req-f773075f-f43e-44d6-af2c-2372fbe61ce0'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.090577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '559646cea52a407e9d71ea5475fa3319', 'request_id': 'req-f773075f-f43e-44d6-af2c-2372fbe61ce0'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.104557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '559646cea52a407e9d71ea5475fa3319', 'request_id': 'req-f773075f-f43e-44d6-af2c-2372fbe61ce0'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.126470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] The token cache is being invalidate because project 559646cea52a407e9d71ea5475fa3319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:42.126855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.127679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.133454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 960/1917] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:41 2022] DELETE /identity/v3/projects/559646cea52a407e9d71ea5475fa3319 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:42.133905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26310a-268c-4563-bf23-e980965c83b4 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:42.149493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26310a-268c-4563-bf23-e980965c83b4 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:42.240436 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:42.262258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3fd89-5f61-4017-95cc-2cfa4167cd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:42.263051 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3fd89-5f61-4017-95cc-2cfa4167cd68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:42.263404 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3fd89-5f61-4017-95cc-2cfa4167cd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:42.263786 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3fd89-5f61-4017-95cc-2cfa4167cd68 admin admin] PATH_INFO: `/v3/users/4f68d8ca6eb44572af56009b360f9c34` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:42.265278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3fd89-5f61-4017-95cc-2cfa4167cd68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f68d8ca6eb44572af56009b360f9c34)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:42.267765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3fd89-5f61-4017-95cc-2cfa4167cd68 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:42.339066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:42.340490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:42.348302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1ec3fd89-5f61-4017-95cc-2cfa4167cd68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f68d8ca6eb44572af56009b360f9c34'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.349242 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1ec3fd89-5f61-4017-95cc-2cfa4167cd68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f68d8ca6eb44572af56009b360f9c34'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.350050 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1ec3fd89-5f61-4017-95cc-2cfa4167cd68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f68d8ca6eb44572af56009b360f9c34'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.360980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773075f-f43e-44d6-af2c-2372fbe61ce0 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:42.366123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:42.367203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:42.367488 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:42.367765 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2 admin admin] PATH_INFO: `/v3/projects/fbbe037cacbb4a0d8925699eb6b5df2c` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:42.369157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbbe037cacbb4a0d8925699eb6b5df2c)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:42.370686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1ec3fd89-5f61-4017-95cc-2cfa4167cd68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f68d8ca6eb44572af56009b360f9c34'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.371195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:42.385058 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1ec3fd89-5f61-4017-95cc-2cfa4167cd68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f68d8ca6eb44572af56009b360f9c34'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.401203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 958/1918] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:42 2022] DELETE /identity/v3/users/4f68d8ca6eb44572af56009b360f9c34 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:42.447477 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbbe037cacbb4a0d8925699eb6b5df2c', 'request_id': 'req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.460021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbbe037cacbb4a0d8925699eb6b5df2c', 'request_id': 'req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.473497 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbbe037cacbb4a0d8925699eb6b5df2c', 'request_id': 'req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.503462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2 admin admin] The token cache is being invalidate because project fbbe037cacbb4a0d8925699eb6b5df2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:42.503830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.504503 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-327a69c0-6ce0-44dd-8c6b-30c0835dbca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:42.512400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 961/1919] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:42 2022] DELETE /identity/v3/projects/fbbe037cacbb4a0d8925699eb6b5df2c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:42.513971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499643f-6496-41ec-b47e-ed49446f66e4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:42.514263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499643f-6496-41ec-b47e-ed49446f66e4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:42.515851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499643f-6496-41ec-b47e-ed49446f66e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:42.555735 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a499643f-6496-41ec-b47e-ed49446f66e4 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:42.558177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a499643f-6496-41ec-b47e-ed49446f66e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:42.640802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 959/1920] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:42.646408 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329f09c-01c2-4dae-8468-31814933ddfb None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:42.646890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329f09c-01c2-4dae-8468-31814933ddfb None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:42.646965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329f09c-01c2-4dae-8468-31814933ddfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:42.677337 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-9329f09c-01c2-4dae-8468-31814933ddfb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:42.679830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9329f09c-01c2-4dae-8468-31814933ddfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:42.689373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 962/1921] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:42.693484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499643f-6496-41ec-b47e-ed49446f66e4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:42.704806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499643f-6496-41ec-b47e-ed49446f66e4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:42.755672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1cd2c-9979-4484-8660-8b801d23bcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:42.756584 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1cd2c-9979-4484-8660-8b801d23bcf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:42.757016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1cd2c-9979-4484-8660-8b801d23bcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:42.757358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1cd2c-9979-4484-8660-8b801d23bcf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:42.758985 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1cd2c-9979-4484-8660-8b801d23bcf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:42.760737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1cd2c-9979-4484-8660-8b801d23bcf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:42.762725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1cd2c-9979-4484-8660-8b801d23bcf4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:42.787520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 960/1922] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:32:42.792622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329f09c-01c2-4dae-8468-31814933ddfb None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:42.803472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329f09c-01c2-4dae-8468-31814933ddfb None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:42.808865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a9122-02dc-439d-8b9f-0883ced84dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:42.809609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a9122-02dc-439d-8b9f-0883ced84dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:42.809890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a9122-02dc-439d-8b9f-0883ced84dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:42.810162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a9122-02dc-439d-8b9f-0883ced84dd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:42.810795 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a9122-02dc-439d-8b9f-0883ced84dd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:42.812690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a9122-02dc-439d-8b9f-0883ced84dd6 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:42.848806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 963/1923] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:42 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:42.852154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1cd2c-9979-4484-8660-8b801d23bcf4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:42.862828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6adca4-c3e6-410a-ac8d-d02e71033922 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:42.863541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6adca4-c3e6-410a-ac8d-d02e71033922 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:42.864005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6adca4-c3e6-410a-ac8d-d02e71033922 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:42.865141 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1cd2c-9979-4484-8660-8b801d23bcf4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:42.868560 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 964/1924] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:32:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:32:42.884358 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bb8d8-2c76-4397-897f-fe48beb47552 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:42.885181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bb8d8-2c76-4397-897f-fe48beb47552 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:42.886330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bb8d8-2c76-4397-897f-fe48beb47552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:42.919774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-381bb8d8-2c76-4397-897f-fe48beb47552 None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:42.919774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-381bb8d8-2c76-4397-897f-fe48beb47552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:42.956640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:42.974200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3587b-751d-4336-aa5f-73593bb26f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:42.977756 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3587b-751d-4336-aa5f-73593bb26f3b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:42.978428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3587b-751d-4336-aa5f-73593bb26f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:42.978928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3587b-751d-4336-aa5f-73593bb26f3b admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:42.994167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3587b-751d-4336-aa5f-73593bb26f3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:42.994167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3587b-751d-4336-aa5f-73593bb26f3b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:43.014377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:43.018809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 965/1925] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:32:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:43.020156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bb8d8-2c76-4397-897f-fe48beb47552 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:43.022463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-381bb8d8-2c76-4397-897f-fe48beb47552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:43.039689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bb8d8-2c76-4397-897f-fe48beb47552 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:43.044615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 961/1926] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:42 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:43.045881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:43.046247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3587b-751d-4336-aa5f-73593bb26f3b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:43.046600 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.046883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.047215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] PATH_INFO: `/v3/projects/f48faf8c8da44ff08d7ec25fcc66b11d` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.048599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48faf8c8da44ff08d7ec25fcc66b11d)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:43.050741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:43.064654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3587b-751d-4336-aa5f-73593bb26f3b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:43.073348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4dbab9-bd90-4009-b015-337fdf212ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:43.074408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4dbab9-bd90-4009-b015-337fdf212ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.075174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4dbab9-bd90-4009-b015-337fdf212ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.075496 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4dbab9-bd90-4009-b015-337fdf212ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.076703 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4dbab9-bd90-4009-b015-337fdf212ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:43.078365 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4dbab9-bd90-4009-b015-337fdf212ce2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:43.093292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 962/1927] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:32:43.094466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4dbab9-bd90-4009-b015-337fdf212ce2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:43.097189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6a4dbab9-bd90-4009-b015-337fdf212ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:43.114790 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48faf8c8da44ff08d7ec25fcc66b11d', 'request_id': 'req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:43.120876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4dbab9-bd90-4009-b015-337fdf212ce2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:43.124136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48faf8c8da44ff08d7ec25fcc66b11d', 'request_id': 'req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:43.135191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48faf8c8da44ff08d7ec25fcc66b11d', 'request_id': 'req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:43.166041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] The token cache is being invalidate because project f48faf8c8da44ff08d7ec25fcc66b11d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:43.166449 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:43.167792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:43.170145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 966/1928] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:43 2022] DELETE /identity/v3/projects/f48faf8c8da44ff08d7ec25fcc66b11d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:43.170963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:43.174279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:43.221838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebb888-a75e-4a58-9f93-b044c0706885 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:43.222221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebb888-a75e-4a58-9f93-b044c0706885 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.222508 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebb888-a75e-4a58-9f93-b044c0706885 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.222790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebb888-a75e-4a58-9f93-b044c0706885 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.225801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ebebb888-a75e-4a58-9f93-b044c0706885 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:43.240323 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebb888-a75e-4a58-9f93-b044c0706885 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:43.242738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebb888-a75e-4a58-9f93-b044c0706885 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:43.256386 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 963/1929] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:43.267072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecaed8-35ff-46b8-af3a-3c23e903d9b6 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:43.268897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2a9df-6d5b-461e-b69f-64f84b626c1a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.269195 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2a9df-6d5b-461e-b69f-64f84b626c1a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.269468 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2a9df-6d5b-461e-b69f-64f84b626c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.272576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7673034-bec6-4b9f-97c3-28f512395ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:43.273871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7673034-bec6-4b9f-97c3-28f512395ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.274322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7673034-bec6-4b9f-97c3-28f512395ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.275273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7673034-bec6-4b9f-97c3-28f512395ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.276911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7673034-bec6-4b9f-97c3-28f512395ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:43.279561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7673034-bec6-4b9f-97c3-28f512395ba4 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:43.294754 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 967/1930] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:43.298816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7673034-bec6-4b9f-97c3-28f512395ba4 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:43.310818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7673034-bec6-4b9f-97c3-28f512395ba4 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:43.311870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-12d2a9df-6d5b-461e-b69f-64f84b626c1a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:43.313553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-12d2a9df-6d5b-461e-b69f-64f84b626c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:43.327482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5631a65-aa2f-41c5-b575-257b483e680c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:43.328390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5631a65-aa2f-41c5-b575-257b483e680c admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.328860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5631a65-aa2f-41c5-b575-257b483e680c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.329276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5631a65-aa2f-41c5-b575-257b483e680c admin admin] PATH_INFO: `/v3/projects/e3a7c71198e54bffb10f299170716843/users/a58a1835a96b4771b5bbf78364bb0527/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.331950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 964/1931] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:43.343674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b2f04-4868-4724-8222-ab5f909ae63d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.344021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b2f04-4868-4724-8222-ab5f909ae63d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.344302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b2f04-4868-4724-8222-ab5f909ae63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.351553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5631a65-aa2f-41c5-b575-257b483e680c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a7c71198e54bffb10f299170716843, user_id=a58a1835a96b4771b5bbf78364bb0527, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:43.356429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5631a65-aa2f-41c5-b575-257b483e680c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:43.387960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 968/1932] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:43 2022] PUT /identity/v3/projects/e3a7c71198e54bffb10f299170716843/users/a58a1835a96b4771b5bbf78364bb0527/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:43.392424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3e3b2f04-4868-4724-8222-ab5f909ae63d None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:43.393715 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3e3b2f04-4868-4724-8222-ab5f909ae63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:43.399383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a6123-e498-44cb-be3a-003ee63f1a6a None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.399673 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a6123-e498-44cb-be3a-003ee63f1a6a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.400495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a6123-e498-44cb-be3a-003ee63f1a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.443527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3f5a6123-e498-44cb-be3a-003ee63f1a6a None None] MFA Rules not processed for user `a58a1835a96b4771b5bbf78364bb0527`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:43.444958 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3f5a6123-e498-44cb-be3a-003ee63f1a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:43.450459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:43.588168 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 965/1933] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:43.589328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b2f04-4868-4724-8222-ab5f909ae63d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:43.604567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 969/1934] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:43.609190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a6123-e498-44cb-be3a-003ee63f1a6a None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:43.612627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b2f04-4868-4724-8222-ab5f909ae63d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:43.623837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a6123-e498-44cb-be3a-003ee63f1a6a None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:43.627083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f33eb-970a-4e53-a349-1e0e5e43de64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:43.628679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f33eb-970a-4e53-a349-1e0e5e43de64 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.629031 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f33eb-970a-4e53-a349-1e0e5e43de64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.629268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f33eb-970a-4e53-a349-1e0e5e43de64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.630837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d5522-2b74-4d79-ac96-8b8241f37377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:43.632307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f33eb-970a-4e53-a349-1e0e5e43de64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:43.632634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f33eb-970a-4e53-a349-1e0e5e43de64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:43.634858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f33eb-970a-4e53-a349-1e0e5e43de64 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:43.637514 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d5522-2b74-4d79-ac96-8b8241f37377 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.638287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d5522-2b74-4d79-ac96-8b8241f37377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.638747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d5522-2b74-4d79-ac96-8b8241f37377 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.639588 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d5522-2b74-4d79-ac96-8b8241f37377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:43.641796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d5522-2b74-4d79-ac96-8b8241f37377 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:43.670209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 966/1935] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:32:43.672108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959821d-0409-4457-ae24-15c04e29e588 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.672108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959821d-0409-4457-ae24-15c04e29e588 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.672345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959821d-0409-4457-ae24-15c04e29e588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.695986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 970/1936] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:43 2022] POST /identity/v3/projects => generated 381 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:43.695986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d5522-2b74-4d79-ac96-8b8241f37377 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:43.709015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8959821d-0409-4457-ae24-15c04e29e588 None None] MFA Rules not processed for user `7536a92033384383bd625b6c969ebd38`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:43.710764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8959821d-0409-4457-ae24-15c04e29e588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:43.802679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:43.818212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d5522-2b74-4d79-ac96-8b8241f37377 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:43.825680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df8ecf-eb72-41d9-aa7d-2546e85e7027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:43.826678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df8ecf-eb72-41d9-aa7d-2546e85e7027 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.827041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df8ecf-eb72-41d9-aa7d-2546e85e7027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.827379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df8ecf-eb72-41d9-aa7d-2546e85e7027 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.828142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df8ecf-eb72-41d9-aa7d-2546e85e7027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:43.840965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 967/1937] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:43.840965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959821d-0409-4457-ae24-15c04e29e588 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:43.850462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959821d-0409-4457-ae24-15c04e29e588 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:43.856291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a64a5-08cf-4d1f-bfbb-29a920bdfb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:43.857004 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a64a5-08cf-4d1f-bfbb-29a920bdfb85 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.857494 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a64a5-08cf-4d1f-bfbb-29a920bdfb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.857773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a64a5-08cf-4d1f-bfbb-29a920bdfb85 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.860500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a64a5-08cf-4d1f-bfbb-29a920bdfb85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:43.860885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df8ecf-eb72-41d9-aa7d-2546e85e7027 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:43.862541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a64a5-08cf-4d1f-bfbb-29a920bdfb85 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:43.904083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 968/1938] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:43 2022] POST /identity/v3/users => generated 297 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:43.905285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a64a5-08cf-4d1f-bfbb-29a920bdfb85 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:43.919296 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a64a5-08cf-4d1f-bfbb-29a920bdfb85 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:43.924495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 971/1939] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:43 2022] POST /identity/v3/projects => generated 409 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:43.927552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245735c6-6e51-469c-9a1d-096424516704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:43.928106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df8ecf-eb72-41d9-aa7d-2546e85e7027 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:43.928953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245735c6-6e51-469c-9a1d-096424516704 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:43.929434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245735c6-6e51-469c-9a1d-096424516704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:43.929901 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245735c6-6e51-469c-9a1d-096424516704 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:43.930837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245735c6-6e51-469c-9a1d-096424516704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:43.934157 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245735c6-6e51-469c-9a1d-096424516704 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:43.955773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 969/1940] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:32:43.957565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245735c6-6e51-469c-9a1d-096424516704 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:43.971514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245735c6-6e51-469c-9a1d-096424516704 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:44.060067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabad9c6-a4ad-4895-af6c-46ef6086db08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:44.060861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabad9c6-a4ad-4895-af6c-46ef6086db08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:44.064853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabad9c6-a4ad-4895-af6c-46ef6086db08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:44.065268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabad9c6-a4ad-4895-af6c-46ef6086db08 admin admin] PATH_INFO: `/v3/projects/41f27dc5d31048848a618a6a2242df90/users/dfb791f4301542ae938b0aa5f034de22/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:44.081358 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df8ecf-eb72-41d9-aa7d-2546e85e7027 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:44.094199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d5acd-9ea6-432b-8e62-34a8809a8607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:44.094433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabad9c6-a4ad-4895-af6c-46ef6086db08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f27dc5d31048848a618a6a2242df90, user_id=dfb791f4301542ae938b0aa5f034de22, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:44.094822 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d5acd-9ea6-432b-8e62-34a8809a8607 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:44.095100 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d5acd-9ea6-432b-8e62-34a8809a8607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:44.095387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d5acd-9ea6-432b-8e62-34a8809a8607 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:44.095985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d5acd-9ea6-432b-8e62-34a8809a8607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:44.098503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabad9c6-a4ad-4895-af6c-46ef6086db08 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:44.111172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d5acd-9ea6-432b-8e62-34a8809a8607 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:44.125685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 970/1941] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:43 2022] PUT /identity/v3/projects/41f27dc5d31048848a618a6a2242df90/users/dfb791f4301542ae938b0aa5f034de22/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:44.133289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabad9c6-a4ad-4895-af6c-46ef6086db08 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:44.137284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:44.150419 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 972/1942] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:43 2022] POST /identity/v3/users => generated 305 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:44.152170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d5acd-9ea6-432b-8e62-34a8809a8607 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:44.176305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d5acd-9ea6-432b-8e62-34a8809a8607 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:44.255329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabad9c6-a4ad-4895-af6c-46ef6086db08 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:44.268319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c311d-700a-4168-9331-3f99c99c197e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:44.269334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c311d-700a-4168-9331-3f99c99c197e admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:44.270098 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c311d-700a-4168-9331-3f99c99c197e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:44.270691 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c311d-700a-4168-9331-3f99c99c197e admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:44.271671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c311d-700a-4168-9331-3f99c99c197e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:44.274113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c311d-700a-4168-9331-3f99c99c197e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:44.276183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aca896-da85-4cca-ad21-396546917c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:44.277113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca896-da85-4cca-ad21-396546917c17 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:44.277388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca896-da85-4cca-ad21-396546917c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:44.277667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca896-da85-4cca-ad21-396546917c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:44.278669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aca896-da85-4cca-ad21-396546917c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:44.280287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aca896-da85-4cca-ad21-396546917c17 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:44.293368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 971/1943] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:44.294519 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 973/1944] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:44.298620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c311d-700a-4168-9331-3f99c99c197e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:44.299848 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aca896-da85-4cca-ad21-396546917c17 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:44.310434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c311d-700a-4168-9331-3f99c99c197e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:44.311879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aca896-da85-4cca-ad21-396546917c17 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:44.316857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b17f8-20e4-4b4e-82aa-b1e2392faf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:44.317508 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b17f8-20e4-4b4e-82aa-b1e2392faf70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:44.317704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df65f2a-1e7d-4f4b-a98c-b5a871286fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:44.317841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b17f8-20e4-4b4e-82aa-b1e2392faf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:44.318046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b17f8-20e4-4b4e-82aa-b1e2392faf70 admin admin] PATH_INFO: `/v3/projects/41f27dc5d31048848a618a6a2242df90/users/dfb791f4301542ae938b0aa5f034de22/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:44.318313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df65f2a-1e7d-4f4b-a98c-b5a871286fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:44.318592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df65f2a-1e7d-4f4b-a98c-b5a871286fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:44.318865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df65f2a-1e7d-4f4b-a98c-b5a871286fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:44.319506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df65f2a-1e7d-4f4b-a98c-b5a871286fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:44.320089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b17f8-20e4-4b4e-82aa-b1e2392faf70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f27dc5d31048848a618a6a2242df90, user_id=dfb791f4301542ae938b0aa5f034de22, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:44.321132 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df65f2a-1e7d-4f4b-a98c-b5a871286fe0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:44.323292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b17f8-20e4-4b4e-82aa-b1e2392faf70 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:44.338430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 974/1945] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:44 2022] PUT /identity/v3/projects/41f27dc5d31048848a618a6a2242df90/users/dfb791f4301542ae938b0aa5f034de22/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:44.342113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 972/1946] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:44.348961 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b17f8-20e4-4b4e-82aa-b1e2392faf70 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:44.351782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d6250-78b4-45f4-8640-b373a986e07e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:44.352019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d6250-78b4-45f4-8640-b373a986e07e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:44.352289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d6250-78b4-45f4-8640-b373a986e07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:44.385381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-195d6250-78b4-45f4-8640-b373a986e07e None None] MFA Rules not processed for user `dfb791f4301542ae938b0aa5f034de22`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:44.386661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-195d6250-78b4-45f4-8640-b373a986e07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:44.406896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:44.426291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b17f8-20e4-4b4e-82aa-b1e2392faf70 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:44.434807 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d0bd4-bd6d-4a5b-aac4-fbd487f8af8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:44.435492 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d0bd4-bd6d-4a5b-aac4-fbd487f8af8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:44.435772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d0bd4-bd6d-4a5b-aac4-fbd487f8af8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:44.436048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d0bd4-bd6d-4a5b-aac4-fbd487f8af8c admin admin] PATH_INFO: `/v3/projects/c7fd331b559244548778b04d930eed21/users/15383830eba240b991207a8035085da4/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:44.458307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d0bd4-bd6d-4a5b-aac4-fbd487f8af8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7fd331b559244548778b04d930eed21, user_id=15383830eba240b991207a8035085da4, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:44.461504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d0bd4-bd6d-4a5b-aac4-fbd487f8af8c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:44.476044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 973/1947] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:44.483533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 975/1948] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:44 2022] PUT /identity/v3/projects/c7fd331b559244548778b04d930eed21/users/15383830eba240b991207a8035085da4/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:44.489013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d6250-78b4-45f4-8640-b373a986e07e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:44.490638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-195d6250-78b4-45f4-8640-b373a986e07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:44.495756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a06226-656b-49e1-83d2-f545794a8b80 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:44.496187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a06226-656b-49e1-83d2-f545794a8b80 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:44.496630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a06226-656b-49e1-83d2-f545794a8b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:44.550742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-25a06226-656b-49e1-83d2-f545794a8b80 None None] MFA Rules not processed for user `15383830eba240b991207a8035085da4`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:44.552059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-25a06226-656b-49e1-83d2-f545794a8b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:44.589481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d6250-78b4-45f4-8640-b373a986e07e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:44.632788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:44.669833 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 976/1949] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:44 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:44.675931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a06226-656b-49e1-83d2-f545794a8b80 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:44.688158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0b4d1-dac5-491b-892e-c82693527a35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:44.688892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0b4d1-dac5-491b-892e-c82693527a35 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:44.689396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0b4d1-dac5-491b-892e-c82693527a35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:44.689980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0b4d1-dac5-491b-892e-c82693527a35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:44.767833 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0b4d1-dac5-491b-892e-c82693527a35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:44.768149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0b4d1-dac5-491b-892e-c82693527a35 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:44.795722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 974/1950] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:44 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:44.814026 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a06226-656b-49e1-83d2-f545794a8b80 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:44.818990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb89d45-7633-4f1f-a113-99e4cb1647dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:44.819775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb89d45-7633-4f1f-a113-99e4cb1647dc admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:44.820203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb89d45-7633-4f1f-a113-99e4cb1647dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:44.820544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb89d45-7633-4f1f-a113-99e4cb1647dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:44.821291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb89d45-7633-4f1f-a113-99e4cb1647dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:44.823359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb89d45-7633-4f1f-a113-99e4cb1647dc admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:44.888517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 977/1951] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:44 2022] POST /identity/v3/projects => generated 407 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:44.892478 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0b4d1-dac5-491b-892e-c82693527a35 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:44.916415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0b4d1-dac5-491b-892e-c82693527a35 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:44.945307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:45.020465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7d100-ba71-4765-ab83-1447ed67e52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:45.021795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7d100-ba71-4765-ab83-1447ed67e52b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:45.022133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7d100-ba71-4765-ab83-1447ed67e52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:45.022498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7d100-ba71-4765-ab83-1447ed67e52b admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:45.023121 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7d100-ba71-4765-ab83-1447ed67e52b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:45.026446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7d100-ba71-4765-ab83-1447ed67e52b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:45.073413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 975/1952] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:44 2022] POST /identity/v3/users => generated 310 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:45.074887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb89d45-7633-4f1f-a113-99e4cb1647dc admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:45.093087 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb89d45-7633-4f1f-a113-99e4cb1647dc admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:45.115836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce421a-7e68-4f23-9c5d-26ddb6fdf158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:45.116613 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce421a-7e68-4f23-9c5d-26ddb6fdf158 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:45.117007 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce421a-7e68-4f23-9c5d-26ddb6fdf158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:45.117365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce421a-7e68-4f23-9c5d-26ddb6fdf158 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:45.118274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce421a-7e68-4f23-9c5d-26ddb6fdf158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:45.121422 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce421a-7e68-4f23-9c5d-26ddb6fdf158 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:45.141397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 978/1953] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:45.142535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7d100-ba71-4765-ab83-1447ed67e52b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:45.161928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7d100-ba71-4765-ab83-1447ed67e52b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:45.174187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e8e01-61e9-4ed9-a5e3-72cac2111efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:45.175224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e8e01-61e9-4ed9-a5e3-72cac2111efa admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:45.175617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e8e01-61e9-4ed9-a5e3-72cac2111efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:45.176000 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e8e01-61e9-4ed9-a5e3-72cac2111efa admin admin] PATH_INFO: `/v3/projects/91cc7930076743a5819c491342ffefee/users/100085b7513e41eb97d12bbf232c6f94/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:45.204741 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e8e01-61e9-4ed9-a5e3-72cac2111efa admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cc7930076743a5819c491342ffefee, user_id=100085b7513e41eb97d12bbf232c6f94, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:45.210598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e8e01-61e9-4ed9-a5e3-72cac2111efa admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:45.246097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 976/1954] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:45 2022] PUT /identity/v3/projects/91cc7930076743a5819c491342ffefee/users/100085b7513e41eb97d12bbf232c6f94/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:45.250095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce421a-7e68-4f23-9c5d-26ddb6fdf158 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:45.281788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce421a-7e68-4f23-9c5d-26ddb6fdf158 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:45.431589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73040f7-ed9a-4ab9-acf4-79bf45bf5bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:45.433701 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73040f7-ed9a-4ab9-acf4-79bf45bf5bea admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:45.435365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73040f7-ed9a-4ab9-acf4-79bf45bf5bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:45.435850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73040f7-ed9a-4ab9-acf4-79bf45bf5bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:45.436702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73040f7-ed9a-4ab9-acf4-79bf45bf5bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:45.442859 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73040f7-ed9a-4ab9-acf4-79bf45bf5bea admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:45.467299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 979/1955] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:45.471330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e8e01-61e9-4ed9-a5e3-72cac2111efa admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:45.486730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e8e01-61e9-4ed9-a5e3-72cac2111efa admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:45.494963 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634090b-96e4-415f-8836-d700ed6911e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:45.495946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634090b-96e4-415f-8836-d700ed6911e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:45.496844 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634090b-96e4-415f-8836-d700ed6911e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:45.497382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634090b-96e4-415f-8836-d700ed6911e7 admin admin] PATH_INFO: `/v3/projects/91cc7930076743a5819c491342ffefee/users/100085b7513e41eb97d12bbf232c6f94/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:45.500425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634090b-96e4-415f-8836-d700ed6911e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cc7930076743a5819c491342ffefee, user_id=100085b7513e41eb97d12bbf232c6f94, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:45.503705 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634090b-96e4-415f-8836-d700ed6911e7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:45.536978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 977/1956] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:45 2022] PUT /identity/v3/projects/91cc7930076743a5819c491342ffefee/users/100085b7513e41eb97d12bbf232c6f94/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:45.555216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4d085-98c0-4758-b5bc-ac529f3e163e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:45.556061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4d085-98c0-4758-b5bc-ac529f3e163e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:45.556441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4d085-98c0-4758-b5bc-ac529f3e163e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:45.605772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-27e4d085-98c0-4758-b5bc-ac529f3e163e None None] MFA Rules not processed for user `100085b7513e41eb97d12bbf232c6f94`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:45.607112 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-27e4d085-98c0-4758-b5bc-ac529f3e163e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:45.684055 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634090b-96e4-415f-8836-d700ed6911e7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:45.688368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:45.711424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 980/1957] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:45 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:45.722654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4d085-98c0-4758-b5bc-ac529f3e163e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:45.724116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-27e4d085-98c0-4758-b5bc-ac529f3e163e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:45.768714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634090b-96e4-415f-8836-d700ed6911e7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:45.773556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:45.774194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:45.774471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:45.774742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] PATH_INFO: `/v3/users/a58a1835a96b4771b5bbf78364bb0527` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:45.775884 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a58a1835a96b4771b5bbf78364bb0527)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:45.777857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:45.787310 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4d085-98c0-4758-b5bc-ac529f3e163e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:45.791680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325715e-82cd-4626-91d0-7bf4ba1cf93e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:45.792292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325715e-82cd-4626-91d0-7bf4ba1cf93e service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:45.792571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325715e-82cd-4626-91d0-7bf4ba1cf93e service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:45.792950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325715e-82cd-4626-91d0-7bf4ba1cf93e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:45.802034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325715e-82cd-4626-91d0-7bf4ba1cf93e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:45.804050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325715e-82cd-4626-91d0-7bf4ba1cf93e service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:45.815826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 981/1958] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:45 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:45.829413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a58a1835a96b4771b5bbf78364bb0527'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:45.830148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a58a1835a96b4771b5bbf78364bb0527'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:45.830826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a58a1835a96b4771b5bbf78364bb0527'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:45.844264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a58a1835a96b4771b5bbf78364bb0527'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:45.853529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a58a1835a96b4771b5bbf78364bb0527'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:45.855443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:45.885601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 978/1959] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:45 2022] DELETE /identity/v3/users/a58a1835a96b4771b5bbf78364bb0527 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:45.890487 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21127cfa-5e1e-445f-bc82-17f8b639c6bd None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:45.890657 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21127cfa-5e1e-445f-bc82-17f8b639c6bd None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:45.890922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21127cfa-5e1e-445f-bc82-17f8b639c6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:45.927841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-21127cfa-5e1e-445f-bc82-17f8b639c6bd None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:45.930341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-21127cfa-5e1e-445f-bc82-17f8b639c6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:46.032205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 982/1960] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:46.061563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:46.063164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:46.094897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21127cfa-5e1e-445f-bc82-17f8b639c6bd None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:46.096341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-21127cfa-5e1e-445f-bc82-17f8b639c6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:46.145861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeeb9ad-c919-42a8-aaf4-30b5066b0abf admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:46.151485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100c968-3474-45ba-b414-b499a3a7b1b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:46.152279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100c968-3474-45ba-b414-b499a3a7b1b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:46.152563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100c968-3474-45ba-b414-b499a3a7b1b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:46.152906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100c968-3474-45ba-b414-b499a3a7b1b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:46.174402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21127cfa-5e1e-445f-bc82-17f8b639c6bd None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:46.177215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100c968-3474-45ba-b414-b499a3a7b1b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:46.178789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a071d-3d98-4e20-b4f2-f1fc0c7c11a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:46.178916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100c968-3474-45ba-b414-b499a3a7b1b6 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:46.179396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a071d-3d98-4e20-b4f2-f1fc0c7c11a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:46.179673 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a071d-3d98-4e20-b4f2-f1fc0c7c11a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:46.179943 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a071d-3d98-4e20-b4f2-f1fc0c7c11a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:46.182591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a99a071d-3d98-4e20-b4f2-f1fc0c7c11a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:46.190172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 979/1961] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:46.248034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:46.267278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a071d-3d98-4e20-b4f2-f1fc0c7c11a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:46.268875 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a071d-3d98-4e20-b4f2-f1fc0c7c11a2 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:46.282381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 983/1962] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:46 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:32:46.503262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100c968-3474-45ba-b414-b499a3a7b1b6 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:46.504946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a100c968-3474-45ba-b414-b499a3a7b1b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:46.527467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100c968-3474-45ba-b414-b499a3a7b1b6 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:46.541353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:46.542235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:46.542770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:46.544226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] PATH_INFO: `/v3/projects/e3a7c71198e54bffb10f299170716843` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:46.547653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a7c71198e54bffb10f299170716843)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:46.549949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:46.661245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a7c71198e54bffb10f299170716843', 'request_id': 'req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:46.674623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a7c71198e54bffb10f299170716843', 'request_id': 'req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:46.686334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a7c71198e54bffb10f299170716843', 'request_id': 'req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:46.717657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] The token cache is being invalidate because project e3a7c71198e54bffb10f299170716843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:46.718155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:46.719392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:46.721646 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 980/1963] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:46 2022] DELETE /identity/v3/projects/e3a7c71198e54bffb10f299170716843 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:46.737027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a071d-3d98-4e20-b4f2-f1fc0c7c11a2 service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:46.753566 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a071d-3d98-4e20-b4f2-f1fc0c7c11a2 service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:46.901521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ac0bf-f0da-416c-9fa1-373e7575d072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:46.902336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ac0bf-f0da-416c-9fa1-373e7575d072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:46.903471 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ac0bf-f0da-416c-9fa1-373e7575d072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:46.903804 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ac0bf-f0da-416c-9fa1-373e7575d072 admin admin] PATH_INFO: `/v3/users/dfb791f4301542ae938b0aa5f034de22` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:46.905045 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ac0bf-f0da-416c-9fa1-373e7575d072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfb791f4301542ae938b0aa5f034de22)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:46.907788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ac0bf-f0da-416c-9fa1-373e7575d072 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:46.983848 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-220ac0bf-f0da-416c-9fa1-373e7575d072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfb791f4301542ae938b0aa5f034de22'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:46.984813 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-220ac0bf-f0da-416c-9fa1-373e7575d072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfb791f4301542ae938b0aa5f034de22'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:46.986335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-220ac0bf-f0da-416c-9fa1-373e7575d072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfb791f4301542ae938b0aa5f034de22'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:46.997425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-220ac0bf-f0da-416c-9fa1-373e7575d072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfb791f4301542ae938b0aa5f034de22'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:47.020935 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-220ac0bf-f0da-416c-9fa1-373e7575d072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfb791f4301542ae938b0aa5f034de22'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:47.042882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 984/1964] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:46 2022] DELETE /identity/v3/users/dfb791f4301542ae938b0aa5f034de22 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:47.472877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:47.474502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:47.488911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ece73e-4032-4eb2-b35e-18ce8fde2e04 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:47.566392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:47.567224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:47.567653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:47.567940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12 admin admin] PATH_INFO: `/v3/projects/41f27dc5d31048848a618a6a2242df90` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:47.569264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f27dc5d31048848a618a6a2242df90)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:47.572947 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:47.603395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:47.637389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f27dc5d31048848a618a6a2242df90', 'request_id': 'req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:47.643765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f27dc5d31048848a618a6a2242df90', 'request_id': 'req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:47.653086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f27dc5d31048848a618a6a2242df90', 'request_id': 'req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:47.668705 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12 admin admin] The token cache is being invalidate because project 41f27dc5d31048848a618a6a2242df90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:47.669289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:47.669914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b4d06ede-05c4-49a8-830a-f9c3dcd1ec12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:47.671676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 981/1965] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:47 2022] DELETE /identity/v3/projects/41f27dc5d31048848a618a6a2242df90 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:47.676338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c42809-27b0-4b45-b56e-efcd8e9c6124 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:47.676699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c42809-27b0-4b45-b56e-efcd8e9c6124 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:47.677092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c42809-27b0-4b45-b56e-efcd8e9c6124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:47.698912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-41c42809-27b0-4b45-b56e-efcd8e9c6124 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:47.700365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-41c42809-27b0-4b45-b56e-efcd8e9c6124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:47.764011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 985/1966] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:47.777332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d62fa9-cadb-4c94-8bc7-747544de315a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:47.777819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d62fa9-cadb-4c94-8bc7-747544de315a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:47.778241 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d62fa9-cadb-4c94-8bc7-747544de315a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:47.807528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f1d62fa9-cadb-4c94-8bc7-747544de315a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:47.808962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f1d62fa9-cadb-4c94-8bc7-747544de315a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:47.820797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 982/1967] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:47.823419 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c42809-27b0-4b45-b56e-efcd8e9c6124 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:47.833838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c42809-27b0-4b45-b56e-efcd8e9c6124 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:47.839549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef281f9f-77fb-472f-9d46-779e9e0a9159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:47.840272 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef281f9f-77fb-472f-9d46-779e9e0a9159 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:47.840652 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef281f9f-77fb-472f-9d46-779e9e0a9159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:47.841048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef281f9f-77fb-472f-9d46-779e9e0a9159 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:47.841922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef281f9f-77fb-472f-9d46-779e9e0a9159 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:47.842394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef281f9f-77fb-472f-9d46-779e9e0a9159 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:47.844194 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef281f9f-77fb-472f-9d46-779e9e0a9159 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:47.846116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:47.869416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d62fa9-cadb-4c94-8bc7-747544de315a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:47.876733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 986/1968] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:32:47.881979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d62fa9-cadb-4c94-8bc7-747544de315a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:47.890291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824e0ac-84e5-4df4-b12f-3c6514a6cb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:47.891113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824e0ac-84e5-4df4-b12f-3c6514a6cb50 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:47.891494 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824e0ac-84e5-4df4-b12f-3c6514a6cb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:47.891868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824e0ac-84e5-4df4-b12f-3c6514a6cb50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:47.892617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824e0ac-84e5-4df4-b12f-3c6514a6cb50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:47.896140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824e0ac-84e5-4df4-b12f-3c6514a6cb50 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:47.955716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef281f9f-77fb-472f-9d46-779e9e0a9159 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:47.965685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 983/1969] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:47 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:32:47.970973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef281f9f-77fb-472f-9d46-779e9e0a9159 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:48.109632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab231fcb-28c5-4f73-bc30-e39845ec8b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:48.110443 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab231fcb-28c5-4f73-bc30-e39845ec8b26 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:48.110851 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab231fcb-28c5-4f73-bc30-e39845ec8b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:48.111258 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab231fcb-28c5-4f73-bc30-e39845ec8b26 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:48.112011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab231fcb-28c5-4f73-bc30-e39845ec8b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:48.117761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab231fcb-28c5-4f73-bc30-e39845ec8b26 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:48.153854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 987/1970] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:47 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:48.158066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824e0ac-84e5-4df4-b12f-3c6514a6cb50 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:48.174179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824e0ac-84e5-4df4-b12f-3c6514a6cb50 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:48.182924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073806c6-9c01-46bc-a75d-32a2c8ed92ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:48.183957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073806c6-9c01-46bc-a75d-32a2c8ed92ec admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:48.184456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073806c6-9c01-46bc-a75d-32a2c8ed92ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:48.185482 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073806c6-9c01-46bc-a75d-32a2c8ed92ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:48.187375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073806c6-9c01-46bc-a75d-32a2c8ed92ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:48.190195 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073806c6-9c01-46bc-a75d-32a2c8ed92ec admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:48.206471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 984/1971] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:48.210353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab231fcb-28c5-4f73-bc30-e39845ec8b26 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:48.222418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab231fcb-28c5-4f73-bc30-e39845ec8b26 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:48.227565 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37191e11-ef46-4a19-924e-520dcfdd3a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:48.228292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37191e11-ef46-4a19-924e-520dcfdd3a4d admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:48.228816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37191e11-ef46-4a19-924e-520dcfdd3a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:48.229698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37191e11-ef46-4a19-924e-520dcfdd3a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:48.229895 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37191e11-ef46-4a19-924e-520dcfdd3a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:48.232085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37191e11-ef46-4a19-924e-520dcfdd3a4d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:48.250235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 988/1972] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:48.254444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073806c6-9c01-46bc-a75d-32a2c8ed92ec admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:48.265420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073806c6-9c01-46bc-a75d-32a2c8ed92ec admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:48.269918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395425b-8326-4a8f-9833-7f36ebe509c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:48.270567 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395425b-8326-4a8f-9833-7f36ebe509c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:48.270849 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395425b-8326-4a8f-9833-7f36ebe509c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:48.271120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395425b-8326-4a8f-9833-7f36ebe509c4 admin admin] PATH_INFO: `/v3/projects/4354ed3558184d18a188a47204281c1f/users/ef92d2a203144c8b830eee743cebde6d/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:48.286700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395425b-8326-4a8f-9833-7f36ebe509c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4354ed3558184d18a188a47204281c1f, user_id=ef92d2a203144c8b830eee743cebde6d, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:48.289448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395425b-8326-4a8f-9833-7f36ebe509c4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:48.303182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 985/1973] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:48 2022] PUT /identity/v3/projects/4354ed3558184d18a188a47204281c1f/users/ef92d2a203144c8b830eee743cebde6d/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:48.315224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a558d-62cb-4fe8-a8c0-936a48e4e226 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:48.315514 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a558d-62cb-4fe8-a8c0-936a48e4e226 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:48.315829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a558d-62cb-4fe8-a8c0-936a48e4e226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:48.344188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-5a6a558d-62cb-4fe8-a8c0-936a48e4e226 None None] MFA Rules not processed for user `ef92d2a203144c8b830eee743cebde6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:48.345437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5a6a558d-62cb-4fe8-a8c0-936a48e4e226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:48.410795 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 989/1974] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:48.415245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395425b-8326-4a8f-9833-7f36ebe509c4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:48.489061 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395425b-8326-4a8f-9833-7f36ebe509c4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:48.494855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dc5d1-4ec8-4819-9f89-a55cb257f133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:48.495563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dc5d1-4ec8-4819-9f89-a55cb257f133 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:48.495848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dc5d1-4ec8-4819-9f89-a55cb257f133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:48.496117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dc5d1-4ec8-4819-9f89-a55cb257f133 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:48.496824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1dc5d1-4ec8-4819-9f89-a55cb257f133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:48.498737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1dc5d1-4ec8-4819-9f89-a55cb257f133 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:48.534613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 986/1975] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:48 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:48.538845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a558d-62cb-4fe8-a8c0-936a48e4e226 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:48.551105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a558d-62cb-4fe8-a8c0-936a48e4e226 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:48.624537 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3724798-e523-43dc-a3a5-edbee30b9051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:48.625225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3724798-e523-43dc-a3a5-edbee30b9051 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:48.625458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3724798-e523-43dc-a3a5-edbee30b9051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:48.625729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3724798-e523-43dc-a3a5-edbee30b9051 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:48.626366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3724798-e523-43dc-a3a5-edbee30b9051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:48.628279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3724798-e523-43dc-a3a5-edbee30b9051 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:48.637779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:48.653857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 990/1976] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:48 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:48.657905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dc5d1-4ec8-4819-9f89-a55cb257f133 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:48.673549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dc5d1-4ec8-4819-9f89-a55cb257f133 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:48.680559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6993ce-b3c2-463c-8214-492d2d2fd7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:48.681748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6993ce-b3c2-463c-8214-492d2d2fd7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:48.682133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6993ce-b3c2-463c-8214-492d2d2fd7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:48.682405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6993ce-b3c2-463c-8214-492d2d2fd7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:48.683054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6993ce-b3c2-463c-8214-492d2d2fd7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:48.684693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6993ce-b3c2-463c-8214-492d2d2fd7c2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:48.695710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 987/1977] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:32:48.700168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3724798-e523-43dc-a3a5-edbee30b9051 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:48.711179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3724798-e523-43dc-a3a5-edbee30b9051 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:48.716156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2de00-83c1-4dce-b9fd-b078869c9ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:48.717088 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2de00-83c1-4dce-b9fd-b078869c9ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:48.717257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2de00-83c1-4dce-b9fd-b078869c9ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:48.717562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2de00-83c1-4dce-b9fd-b078869c9ae0 admin admin] PATH_INFO: `/v3/projects/462543c2196743819de0f7363ce9f425/users/844bba059ba6421db8fdedab4443fbc7/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:48.733927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2de00-83c1-4dce-b9fd-b078869c9ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=462543c2196743819de0f7363ce9f425, user_id=844bba059ba6421db8fdedab4443fbc7, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:48.736587 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2de00-83c1-4dce-b9fd-b078869c9ae0 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:48.750125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 991/1978] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:48 2022] PUT /identity/v3/projects/462543c2196743819de0f7363ce9f425/users/844bba059ba6421db8fdedab4443fbc7/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:48.754306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6993ce-b3c2-463c-8214-492d2d2fd7c2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:48.764467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6993ce-b3c2-463c-8214-492d2d2fd7c2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:48.777507 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:48.824320 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e487534-5f36-4d04-aac2-ec6dd6436877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:48.824965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e487534-5f36-4d04-aac2-ec6dd6436877 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:48.825246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e487534-5f36-4d04-aac2-ec6dd6436877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:48.825518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e487534-5f36-4d04-aac2-ec6dd6436877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:48.826158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e487534-5f36-4d04-aac2-ec6dd6436877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:48.827789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e487534-5f36-4d04-aac2-ec6dd6436877 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:48.838053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 988/1979] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:48.842380 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2de00-83c1-4dce-b9fd-b078869c9ae0 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:48.853543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2de00-83c1-4dce-b9fd-b078869c9ae0 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:48.858161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0b598-e86d-40b7-b0e0-8c2e80ba9f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:48.858820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0b598-e86d-40b7-b0e0-8c2e80ba9f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:48.859098 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0b598-e86d-40b7-b0e0-8c2e80ba9f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:48.859373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0b598-e86d-40b7-b0e0-8c2e80ba9f34 admin admin] PATH_INFO: `/v3/projects/462543c2196743819de0f7363ce9f425/users/844bba059ba6421db8fdedab4443fbc7/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:48.861312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0b598-e86d-40b7-b0e0-8c2e80ba9f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=462543c2196743819de0f7363ce9f425, user_id=844bba059ba6421db8fdedab4443fbc7, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:48.863986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0b598-e86d-40b7-b0e0-8c2e80ba9f34 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:48.875032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 992/1980] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:48 2022] PUT /identity/v3/projects/462543c2196743819de0f7363ce9f425/users/844bba059ba6421db8fdedab4443fbc7/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:48.885789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab6879-502a-47dd-82e8-0aba51bcb2b0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:48.886087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab6879-502a-47dd-82e8-0aba51bcb2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:48.886368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab6879-502a-47dd-82e8-0aba51bcb2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:48.915721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-57ab6879-502a-47dd-82e8-0aba51bcb2b0 None None] MFA Rules not processed for user `844bba059ba6421db8fdedab4443fbc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:48.917287 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-57ab6879-502a-47dd-82e8-0aba51bcb2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:48.988924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 989/1981] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:32:49.000809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0b598-e86d-40b7-b0e0-8c2e80ba9f34 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:49.002976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9eb0b598-e86d-40b7-b0e0-8c2e80ba9f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:49.088370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0b598-e86d-40b7-b0e0-8c2e80ba9f34 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:49.092071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ce5ca-7ea3-46a0-9a47-1bc466f6c3a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:49.092700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ce5ca-7ea3-46a0-9a47-1bc466f6c3a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:49.093015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ce5ca-7ea3-46a0-9a47-1bc466f6c3a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:49.093289 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ce5ca-7ea3-46a0-9a47-1bc466f6c3a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:49.173270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ce5ca-7ea3-46a0-9a47-1bc466f6c3a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:49.174884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ce5ca-7ea3-46a0-9a47-1bc466f6c3a6 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:49.190290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 993/1982] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:54.607966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab6879-502a-47dd-82e8-0aba51bcb2b0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:54.646859 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab6879-502a-47dd-82e8-0aba51bcb2b0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:54.665370 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab8dc2-cf97-42a3-bef7-35af2b461c04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:54.667066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab8dc2-cf97-42a3-bef7-35af2b461c04 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:54.667066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab8dc2-cf97-42a3-bef7-35af2b461c04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:54.667066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab8dc2-cf97-42a3-bef7-35af2b461c04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:54.701725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab8dc2-cf97-42a3-bef7-35af2b461c04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:54.703211 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab8dc2-cf97-42a3-bef7-35af2b461c04 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:54.721922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 990/1983] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:55.183280 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ce5ca-7ea3-46a0-9a47-1bc466f6c3a6 service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:55.184907 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6b8ce5ca-7ea3-46a0-9a47-1bc466f6c3a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:55.350951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ce5ca-7ea3-46a0-9a47-1bc466f6c3a6 service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:55.357202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:55.358009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:55.358388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:55.358866 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] PATH_INFO: `/v3/users/15383830eba240b991207a8035085da4` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:55.360910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15383830eba240b991207a8035085da4)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:55.363058 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:55.379252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:55.462972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15383830eba240b991207a8035085da4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:55.463856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15383830eba240b991207a8035085da4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:55.464377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15383830eba240b991207a8035085da4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:55.483167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15383830eba240b991207a8035085da4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:55.500464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15383830eba240b991207a8035085da4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:55.520800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 994/1984] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:55 2022] DELETE /identity/v3/users/15383830eba240b991207a8035085da4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:55.531246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785201a4-90c6-4253-83b0-789095d64f43 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:55.531525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785201a4-90c6-4253-83b0-789095d64f43 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:55.531811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785201a4-90c6-4253-83b0-789095d64f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:55.554622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-785201a4-90c6-4253-83b0-789095d64f43 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:55.557652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-785201a4-90c6-4253-83b0-789095d64f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:55.644734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 991/1985] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:55.695010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:55.696672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:55.816053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b46bf-0b7a-4b3a-81ee-ab4ff28406d9 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:55.819517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a284d71-ef11-4926-aad3-d71f1824340c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:55.820505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a284d71-ef11-4926-aad3-d71f1824340c service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:55.820505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a284d71-ef11-4926-aad3-d71f1824340c service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:55.820699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a284d71-ef11-4926-aad3-d71f1824340c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:55.830936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a284d71-ef11-4926-aad3-d71f1824340c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:55.832416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a284d71-ef11-4926-aad3-d71f1824340c service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:55.842872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 995/1986] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:56.294015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785201a4-90c6-4253-83b0-789095d64f43 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:56.295662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-785201a4-90c6-4253-83b0-789095d64f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:56.331182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785201a4-90c6-4253-83b0-789095d64f43 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:56.336345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:56.337169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:56.337447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:56.337730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] PATH_INFO: `/v3/projects/c7fd331b559244548778b04d930eed21` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:56.338992 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7fd331b559244548778b04d930eed21)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:56.341173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:56.367592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:56.410035 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7fd331b559244548778b04d930eed21', 'request_id': 'req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:56.410035 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7fd331b559244548778b04d930eed21', 'request_id': 'req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:56.412698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7fd331b559244548778b04d930eed21', 'request_id': 'req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:56.427196 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] The token cache is being invalidate because project c7fd331b559244548778b04d930eed21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:56.427498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:56.428298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:56.431418 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 992/1987] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:56 2022] DELETE /identity/v3/projects/c7fd331b559244548778b04d930eed21 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:56.435145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a284d71-ef11-4926-aad3-d71f1824340c service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:56.449695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a284d71-ef11-4926-aad3-d71f1824340c service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:56.524952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9c9c5-283b-45dc-90a9-72fe742b2247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:56.524952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9c9c5-283b-45dc-90a9-72fe742b2247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:56.524952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9c9c5-283b-45dc-90a9-72fe742b2247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:56.524952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9c9c5-283b-45dc-90a9-72fe742b2247 admin admin] PATH_INFO: `/v3/users/100085b7513e41eb97d12bbf232c6f94` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:56.525287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9c9c5-283b-45dc-90a9-72fe742b2247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=100085b7513e41eb97d12bbf232c6f94)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:56.525287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9c9c5-283b-45dc-90a9-72fe742b2247 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:56.563577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:56.582159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-05d9c9c5-283b-45dc-90a9-72fe742b2247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100085b7513e41eb97d12bbf232c6f94'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:56.583283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-05d9c9c5-283b-45dc-90a9-72fe742b2247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100085b7513e41eb97d12bbf232c6f94'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:56.583902 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-05d9c9c5-283b-45dc-90a9-72fe742b2247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100085b7513e41eb97d12bbf232c6f94'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:56.594389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-05d9c9c5-283b-45dc-90a9-72fe742b2247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100085b7513e41eb97d12bbf232c6f94'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:56.609132 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-05d9c9c5-283b-45dc-90a9-72fe742b2247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100085b7513e41eb97d12bbf232c6f94'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:56.634754 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 996/1988] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:32:56 2022] DELETE /identity/v3/users/100085b7513e41eb97d12bbf232c6f94 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:57.080708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:57.082621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:57.119197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ed002-9013-46ac-a7cf-35ef8d4e6b91 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:57.190067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed4db1-151a-414c-a464-8f3ebf27bff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:57.190724 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed4db1-151a-414c-a464-8f3ebf27bff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:57.191009 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed4db1-151a-414c-a464-8f3ebf27bff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:57.191312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed4db1-151a-414c-a464-8f3ebf27bff2 admin admin] PATH_INFO: `/v3/projects/91cc7930076743a5819c491342ffefee` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:57.192474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed4db1-151a-414c-a464-8f3ebf27bff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91cc7930076743a5819c491342ffefee)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:57.194775 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed4db1-151a-414c-a464-8f3ebf27bff2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:57.248307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f3ed4db1-151a-414c-a464-8f3ebf27bff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91cc7930076743a5819c491342ffefee', 'request_id': 'req-f3ed4db1-151a-414c-a464-8f3ebf27bff2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:57.254483 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f3ed4db1-151a-414c-a464-8f3ebf27bff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91cc7930076743a5819c491342ffefee', 'request_id': 'req-f3ed4db1-151a-414c-a464-8f3ebf27bff2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:57.262285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f3ed4db1-151a-414c-a464-8f3ebf27bff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91cc7930076743a5819c491342ffefee', 'request_id': 'req-f3ed4db1-151a-414c-a464-8f3ebf27bff2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:57.277210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f3ed4db1-151a-414c-a464-8f3ebf27bff2 admin admin] The token cache is being invalidate because project 91cc7930076743a5819c491342ffefee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:32:57.277539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f3ed4db1-151a-414c-a464-8f3ebf27bff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:57.278169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f3ed4db1-151a-414c-a464-8f3ebf27bff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:32:57.279655 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 993/1989] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:32:57 2022] DELETE /identity/v3/projects/91cc7930076743a5819c491342ffefee => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:57.284854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8c3d0-5181-4b85-8e6f-83516c5a0494 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:57.285281 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8c3d0-5181-4b85-8e6f-83516c5a0494 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:57.285662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8c3d0-5181-4b85-8e6f-83516c5a0494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:57.310292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d7a8c3d0-5181-4b85-8e6f-83516c5a0494 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:57.311840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d7a8c3d0-5181-4b85-8e6f-83516c5a0494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:57.380433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 997/1990] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:57.388337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af54a15-a68a-47c9-88bf-e3ed44c759d0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:57.388624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af54a15-a68a-47c9-88bf-e3ed44c759d0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:57.388944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af54a15-a68a-47c9-88bf-e3ed44c759d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:57.409383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5af54a15-a68a-47c9-88bf-e3ed44c759d0 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:57.410593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5af54a15-a68a-47c9-88bf-e3ed44c759d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:57.419266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 994/1991] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:57.423510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8c3d0-5181-4b85-8e6f-83516c5a0494 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:57.438416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8c3d0-5181-4b85-8e6f-83516c5a0494 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:57.443475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f7b79-feeb-4f5d-b0b0-da4acd9b2643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:57.444293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f7b79-feeb-4f5d-b0b0-da4acd9b2643 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:57.444581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f7b79-feeb-4f5d-b0b0-da4acd9b2643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:57.444889 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f7b79-feeb-4f5d-b0b0-da4acd9b2643 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:57.445550 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f7b79-feeb-4f5d-b0b0-da4acd9b2643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:32:57.445887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f7b79-feeb-4f5d-b0b0-da4acd9b2643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:57.448045 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f7b79-feeb-4f5d-b0b0-da4acd9b2643 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:57.460604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 998/1992] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:32:57.465842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af54a15-a68a-47c9-88bf-e3ed44c759d0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:57.475284 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af54a15-a68a-47c9-88bf-e3ed44c759d0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:57.481695 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b30797-cee7-42e8-a0dc-3d0f05651850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:57.482322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b30797-cee7-42e8-a0dc-3d0f05651850 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:57.482604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b30797-cee7-42e8-a0dc-3d0f05651850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:57.482878 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b30797-cee7-42e8-a0dc-3d0f05651850 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:57.483541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b30797-cee7-42e8-a0dc-3d0f05651850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:57.485539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b30797-cee7-42e8-a0dc-3d0f05651850 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:57.518390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 995/1993] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:57 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:57.519667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f7b79-feeb-4f5d-b0b0-da4acd9b2643 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:57.530371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f7b79-feeb-4f5d-b0b0-da4acd9b2643 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:57.586989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a6c3d-f5bd-41a7-9fd0-8c8d0f1e12ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:57.587597 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a6c3d-f5bd-41a7-9fd0-8c8d0f1e12ed admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:57.587878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a6c3d-f5bd-41a7-9fd0-8c8d0f1e12ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:57.588149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a6c3d-f5bd-41a7-9fd0-8c8d0f1e12ed admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:57.588795 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a6c3d-f5bd-41a7-9fd0-8c8d0f1e12ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:57.590720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a6c3d-f5bd-41a7-9fd0-8c8d0f1e12ed admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:57.611697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 999/1994] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:57 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:57.615944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b30797-cee7-42e8-a0dc-3d0f05651850 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:57.626896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b30797-cee7-42e8-a0dc-3d0f05651850 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:57.631462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0e3ec-5803-4448-8180-f64c7d50eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:57.632057 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0e3ec-5803-4448-8180-f64c7d50eb0a admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:57.632340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0e3ec-5803-4448-8180-f64c7d50eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:57.632615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0e3ec-5803-4448-8180-f64c7d50eb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:57.633294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0e3ec-5803-4448-8180-f64c7d50eb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:57.635621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0e3ec-5803-4448-8180-f64c7d50eb0a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:57.641236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:57.647010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 996/1995] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:57.650953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a6c3d-f5bd-41a7-9fd0-8c8d0f1e12ed admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:57.661143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a6c3d-f5bd-41a7-9fd0-8c8d0f1e12ed admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:57.666548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358e5cf-961f-4eee-b0cf-bb81221a37dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:57.667176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358e5cf-961f-4eee-b0cf-bb81221a37dc admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:57.667433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358e5cf-961f-4eee-b0cf-bb81221a37dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:57.667708 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358e5cf-961f-4eee-b0cf-bb81221a37dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:57.668348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358e5cf-961f-4eee-b0cf-bb81221a37dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:57.669986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358e5cf-961f-4eee-b0cf-bb81221a37dc admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:57.681340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1000/1996] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:32:57.685879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0e3ec-5803-4448-8180-f64c7d50eb0a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:57.696166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0e3ec-5803-4448-8180-f64c7d50eb0a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:57.700727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48444035-faed-431a-94d6-717fdb0d4da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:57.701413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48444035-faed-431a-94d6-717fdb0d4da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:57.701694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48444035-faed-431a-94d6-717fdb0d4da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:57.701965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48444035-faed-431a-94d6-717fdb0d4da1 admin admin] PATH_INFO: `/v3/projects/07a31333a2f14aeca6cc20739c72bbc3/users/ebb681992237466db91cc3df29c90f09/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:57.717161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48444035-faed-431a-94d6-717fdb0d4da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07a31333a2f14aeca6cc20739c72bbc3, user_id=ebb681992237466db91cc3df29c90f09, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:57.719846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48444035-faed-431a-94d6-717fdb0d4da1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:57.734684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 997/1997] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:57 2022] PUT /identity/v3/projects/07a31333a2f14aeca6cc20739c72bbc3/users/ebb681992237466db91cc3df29c90f09/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:32:57.739835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6218b0-bde9-493a-9038-2fe428657e55 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:57.740125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6218b0-bde9-493a-9038-2fe428657e55 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:57.740399 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6218b0-bde9-493a-9038-2fe428657e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:57.759727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-7a6218b0-bde9-493a-9038-2fe428657e55 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:57.760978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7a6218b0-bde9-493a-9038-2fe428657e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:57.770191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:57.827560 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1001/1998] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:57.837148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48444035-faed-431a-94d6-717fdb0d4da1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:57.838855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-48444035-faed-431a-94d6-717fdb0d4da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:57.899771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48444035-faed-431a-94d6-717fdb0d4da1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:57.904324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651e8d8-06d3-4b47-a940-e274e94af218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:57.905310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651e8d8-06d3-4b47-a940-e274e94af218 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:57.905763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651e8d8-06d3-4b47-a940-e274e94af218 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:57.907072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651e8d8-06d3-4b47-a940-e274e94af218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:57.918113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651e8d8-06d3-4b47-a940-e274e94af218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:57.920513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651e8d8-06d3-4b47-a940-e274e94af218 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:57.933132 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 998/1999] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:32:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:32:59.045169 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d853e-cba5-4f9f-a937-c47cda805052 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:59.046008 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d853e-cba5-4f9f-a937-c47cda805052 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:59.046295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d853e-cba5-4f9f-a937-c47cda805052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:59.114687 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-015d853e-cba5-4f9f-a937-c47cda805052 None None] MFA Rules not processed for user `ebb681992237466db91cc3df29c90f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:32:59.119079 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-015d853e-cba5-4f9f-a937-c47cda805052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:32:59.298606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1002/2000] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:32:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:32:59.300698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651e8d8-06d3-4b47-a940-e274e94af218 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:59.318544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651e8d8-06d3-4b47-a940-e274e94af218 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:59.328719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d4073-ef31-4fb8-b081-7ff78156c209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:59.329510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d4073-ef31-4fb8-b081-7ff78156c209 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:59.329868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d4073-ef31-4fb8-b081-7ff78156c209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:59.330383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d4073-ef31-4fb8-b081-7ff78156c209 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:59.334763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d4073-ef31-4fb8-b081-7ff78156c209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:59.340965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d4073-ef31-4fb8-b081-7ff78156c209 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:59.376787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 999/2001] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:32:59 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:59.378949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d853e-cba5-4f9f-a937-c47cda805052 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:59.391121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d853e-cba5-4f9f-a937-c47cda805052 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:59.453716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33600c-00a3-4bea-8915-88f370316af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:59.454337 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33600c-00a3-4bea-8915-88f370316af1 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:59.454617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33600c-00a3-4bea-8915-88f370316af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:59.454892 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33600c-00a3-4bea-8915-88f370316af1 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:59.455678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33600c-00a3-4bea-8915-88f370316af1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:59.457634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33600c-00a3-4bea-8915-88f370316af1 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:59.482404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1003/2002] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:32:59 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:32:59.486600 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d4073-ef31-4fb8-b081-7ff78156c209 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:59.497245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d4073-ef31-4fb8-b081-7ff78156c209 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:59.502182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3ad25-f01a-45f2-9f9d-f59ed731db8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:59.502890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3ad25-f01a-45f2-9f9d-f59ed731db8f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:59.503243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3ad25-f01a-45f2-9f9d-f59ed731db8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:59.503583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3ad25-f01a-45f2-9f9d-f59ed731db8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:59.504396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3ad25-f01a-45f2-9f9d-f59ed731db8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:59.506459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3ad25-f01a-45f2-9f9d-f59ed731db8f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:59.518624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1000/2003] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:59.522980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33600c-00a3-4bea-8915-88f370316af1 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:59.550183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33600c-00a3-4bea-8915-88f370316af1 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:59.556674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e2002-85b2-4aff-b302-8a6e280fcebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:59.557656 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e2002-85b2-4aff-b302-8a6e280fcebf admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:59.558021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e2002-85b2-4aff-b302-8a6e280fcebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:59.558302 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e2002-85b2-4aff-b302-8a6e280fcebf admin admin] PATH_INFO: `/v3/projects/de32071c5b2346ff85920301f838d296/users/5da568a7163b4535a4dca6fbe038bea3/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:59.578076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e2002-85b2-4aff-b302-8a6e280fcebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=de32071c5b2346ff85920301f838d296, user_id=5da568a7163b4535a4dca6fbe038bea3, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:59.581309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e2002-85b2-4aff-b302-8a6e280fcebf admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:59.600838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1004/2004] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:59 2022] PUT /identity/v3/projects/de32071c5b2346ff85920301f838d296/users/5da568a7163b4535a4dca6fbe038bea3/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:32:59.604584 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3ad25-f01a-45f2-9f9d-f59ed731db8f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:59.622729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3ad25-f01a-45f2-9f9d-f59ed731db8f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:59.706607 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402be403-8361-4951-83ca-46204562401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:59.707372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402be403-8361-4951-83ca-46204562401b admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:59.707753 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402be403-8361-4951-83ca-46204562401b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:59.708145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402be403-8361-4951-83ca-46204562401b admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:59.709282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402be403-8361-4951-83ca-46204562401b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:59.710837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402be403-8361-4951-83ca-46204562401b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:59.721929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1001/2005] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:32:59.725973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e2002-85b2-4aff-b302-8a6e280fcebf admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:32:59.736639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e2002-85b2-4aff-b302-8a6e280fcebf admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:32:59.741356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7ea2f-1997-47b7-8eed-4a7541f0bf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:32:59.742013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7ea2f-1997-47b7-8eed-4a7541f0bf68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:32:59.742315 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7ea2f-1997-47b7-8eed-4a7541f0bf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:32:59.743108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7ea2f-1997-47b7-8eed-4a7541f0bf68 admin admin] PATH_INFO: `/v3/projects/de32071c5b2346ff85920301f838d296/users/5da568a7163b4535a4dca6fbe038bea3/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:32:59.745176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7ea2f-1997-47b7-8eed-4a7541f0bf68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de32071c5b2346ff85920301f838d296, user_id=5da568a7163b4535a4dca6fbe038bea3, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:32:59.747822 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7ea2f-1997-47b7-8eed-4a7541f0bf68 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:32:59.754041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:32:59.762339 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1005/2006] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:32:59 2022] PUT /identity/v3/projects/de32071c5b2346ff85920301f838d296/users/5da568a7163b4535a4dca6fbe038bea3/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:00.309897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402be403-8361-4951-83ca-46204562401b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:00.312959 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-402be403-8361-4951-83ca-46204562401b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:00.331083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402be403-8361-4951-83ca-46204562401b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:00.380251 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:00.398425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:00.399279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:00.399918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:00.400046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] PATH_INFO: `/v3/users/ef92d2a203144c8b830eee743cebde6d` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:00.401399 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef92d2a203144c8b830eee743cebde6d)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:00.404740 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:00.474116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef92d2a203144c8b830eee743cebde6d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:00.476346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef92d2a203144c8b830eee743cebde6d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:00.477486 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef92d2a203144c8b830eee743cebde6d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:00.509448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef92d2a203144c8b830eee743cebde6d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:00.520313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef92d2a203144c8b830eee743cebde6d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:00.543854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1002/2007] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:33:00 2022] DELETE /identity/v3/users/ef92d2a203144c8b830eee743cebde6d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:00.549067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407511a-57fc-4d70-902d-c943d52f1a24 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:00.549546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407511a-57fc-4d70-902d-c943d52f1a24 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:00.550365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407511a-57fc-4d70-902d-c943d52f1a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:00.570828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b407511a-57fc-4d70-902d-c943d52f1a24 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:00.572872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b407511a-57fc-4d70-902d-c943d52f1a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:00.634888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1006/2008] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:00.645513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:00.647940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:00.710604 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c403ab-1f6c-4da4-bd09-7291036de4cb admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:00.714958 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f5198-5877-4f06-a1ae-dd5b17dcbd4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:00.715573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f5198-5877-4f06-a1ae-dd5b17dcbd4d service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:00.715936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f5198-5877-4f06-a1ae-dd5b17dcbd4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:00.716211 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f5198-5877-4f06-a1ae-dd5b17dcbd4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:00.730851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f5198-5877-4f06-a1ae-dd5b17dcbd4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:00.732944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f5198-5877-4f06-a1ae-dd5b17dcbd4d service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:00.745991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1003/2009] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:01.341001 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407511a-57fc-4d70-902d-c943d52f1a24 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:01.342563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b407511a-57fc-4d70-902d-c943d52f1a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:01.357913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407511a-57fc-4d70-902d-c943d52f1a24 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:01.366098 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a8b3a-0993-430c-b876-ca29914f5101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:01.368199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a8b3a-0993-430c-b876-ca29914f5101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:01.368841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a8b3a-0993-430c-b876-ca29914f5101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:01.369201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a8b3a-0993-430c-b876-ca29914f5101 admin admin] PATH_INFO: `/v3/projects/4354ed3558184d18a188a47204281c1f` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:01.370670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a8b3a-0993-430c-b876-ca29914f5101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4354ed3558184d18a188a47204281c1f)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:01.372740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a8b3a-0993-430c-b876-ca29914f5101 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:01.447334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ab7a8b3a-0993-430c-b876-ca29914f5101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4354ed3558184d18a188a47204281c1f', 'request_id': 'req-ab7a8b3a-0993-430c-b876-ca29914f5101'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:01.455108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ab7a8b3a-0993-430c-b876-ca29914f5101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4354ed3558184d18a188a47204281c1f', 'request_id': 'req-ab7a8b3a-0993-430c-b876-ca29914f5101'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:01.461316 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ab7a8b3a-0993-430c-b876-ca29914f5101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4354ed3558184d18a188a47204281c1f', 'request_id': 'req-ab7a8b3a-0993-430c-b876-ca29914f5101'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:01.475405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ab7a8b3a-0993-430c-b876-ca29914f5101 admin admin] The token cache is being invalidate because project 4354ed3558184d18a188a47204281c1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:33:01.475700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ab7a8b3a-0993-430c-b876-ca29914f5101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:01.476328 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ab7a8b3a-0993-430c-b876-ca29914f5101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:01.482255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1007/2010] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:33:01 2022] DELETE /identity/v3/projects/4354ed3558184d18a188a47204281c1f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:01.484782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f5198-5877-4f06-a1ae-dd5b17dcbd4d service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:01.497688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f5198-5877-4f06-a1ae-dd5b17dcbd4d service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:01.564918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:01.565557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:01.565862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:01.566137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] PATH_INFO: `/v3/users/844bba059ba6421db8fdedab4443fbc7` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:01.567280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=844bba059ba6421db8fdedab4443fbc7)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:01.569529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:01.574477 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:01.619532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '844bba059ba6421db8fdedab4443fbc7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:01.620189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '844bba059ba6421db8fdedab4443fbc7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:01.620966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '844bba059ba6421db8fdedab4443fbc7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:01.635909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '844bba059ba6421db8fdedab4443fbc7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:01.668095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '844bba059ba6421db8fdedab4443fbc7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:01.700860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1004/2011] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:33:01 2022] DELETE /identity/v3/users/844bba059ba6421db8fdedab4443fbc7 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:01.744587 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c90d00-76b7-4db3-aa3d-ba3c1e06bc56 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:01.745580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c90d00-76b7-4db3-aa3d-ba3c1e06bc56 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:01.746057 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c90d00-76b7-4db3-aa3d-ba3c1e06bc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:01.785060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-97c90d00-76b7-4db3-aa3d-ba3c1e06bc56 None None] MFA Rules not processed for user `5da568a7163b4535a4dca6fbe038bea3`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:01.787249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-97c90d00-76b7-4db3-aa3d-ba3c1e06bc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:02.022751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1008/2012] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:02.029877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:02.031722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:02.204881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3df3c4-a0ed-4a15-bfd8-ebe77d16b484 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:02.215804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab364e-d2af-4f1a-a271-4eb445a6de97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:02.216460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab364e-d2af-4f1a-a271-4eb445a6de97 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:02.216981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab364e-d2af-4f1a-a271-4eb445a6de97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:02.217466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab364e-d2af-4f1a-a271-4eb445a6de97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:02.219711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d6ab364e-d2af-4f1a-a271-4eb445a6de97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:02.295756 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab364e-d2af-4f1a-a271-4eb445a6de97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:02.297881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab364e-d2af-4f1a-a271-4eb445a6de97 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:02.309708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1005/2013] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:33:02.693323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c90d00-76b7-4db3-aa3d-ba3c1e06bc56 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:02.694021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-97c90d00-76b7-4db3-aa3d-ba3c1e06bc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:02.719945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:02.817166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c90d00-76b7-4db3-aa3d-ba3c1e06bc56 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:02.823423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fbad6-0af2-4c52-a3c6-e452344a1686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:02.824535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fbad6-0af2-4c52-a3c6-e452344a1686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:02.825047 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fbad6-0af2-4c52-a3c6-e452344a1686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:02.825542 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fbad6-0af2-4c52-a3c6-e452344a1686 admin admin] PATH_INFO: `/v3/projects/462543c2196743819de0f7363ce9f425` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:02.827077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fbad6-0af2-4c52-a3c6-e452344a1686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=462543c2196743819de0f7363ce9f425)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:02.829291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fbad6-0af2-4c52-a3c6-e452344a1686 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:02.898542 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-a57fbad6-0af2-4c52-a3c6-e452344a1686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '462543c2196743819de0f7363ce9f425', 'request_id': 'req-a57fbad6-0af2-4c52-a3c6-e452344a1686'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:02.908066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-a57fbad6-0af2-4c52-a3c6-e452344a1686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '462543c2196743819de0f7363ce9f425', 'request_id': 'req-a57fbad6-0af2-4c52-a3c6-e452344a1686'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:02.915684 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-a57fbad6-0af2-4c52-a3c6-e452344a1686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '462543c2196743819de0f7363ce9f425', 'request_id': 'req-a57fbad6-0af2-4c52-a3c6-e452344a1686'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:02.936792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-a57fbad6-0af2-4c52-a3c6-e452344a1686 admin admin] The token cache is being invalidate because project 462543c2196743819de0f7363ce9f425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:33:02.936792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-a57fbad6-0af2-4c52-a3c6-e452344a1686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:02.936792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-a57fbad6-0af2-4c52-a3c6-e452344a1686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:02.937154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1009/2014] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:33:02 2022] DELETE /identity/v3/projects/462543c2196743819de0f7363ce9f425 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:02.943011 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172574c1-4961-4ab5-8259-dd0f724e9ac6 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:02.944994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172574c1-4961-4ab5-8259-dd0f724e9ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:02.945405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172574c1-4961-4ab5-8259-dd0f724e9ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:02.979802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-172574c1-4961-4ab5-8259-dd0f724e9ac6 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:02.981377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-172574c1-4961-4ab5-8259-dd0f724e9ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:03.029823 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:03.068717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1006/2015] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:33:03.079075 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fbbb6-4d0d-4e33-92b9-c8f62147d056 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:03.079381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fbbb6-4d0d-4e33-92b9-c8f62147d056 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:03.079659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fbbb6-4d0d-4e33-92b9-c8f62147d056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:03.103973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-674fbbb6-4d0d-4e33-92b9-c8f62147d056 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:03.105525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-674fbbb6-4d0d-4e33-92b9-c8f62147d056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:03.117531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1010/2016] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:03.125187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172574c1-4961-4ab5-8259-dd0f724e9ac6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:03.137325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172574c1-4961-4ab5-8259-dd0f724e9ac6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:03.142525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6bbfb-d101-42ed-85c0-304ce21da8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:03.143249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6bbfb-d101-42ed-85c0-304ce21da8ac admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:03.143597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6bbfb-d101-42ed-85c0-304ce21da8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:03.143937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6bbfb-d101-42ed-85c0-304ce21da8ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:03.144667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6bbfb-d101-42ed-85c0-304ce21da8ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:33:03.145086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6bbfb-d101-42ed-85c0-304ce21da8ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:03.146770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6bbfb-d101-42ed-85c0-304ce21da8ac admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:03.172021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fbbb6-4d0d-4e33-92b9-c8f62147d056 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:03.172962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1007/2017] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:33:03.182767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fbbb6-4d0d-4e33-92b9-c8f62147d056 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:03.188114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504fe14-8d54-4a12-aeaa-c810fb988a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:03.188485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504fe14-8d54-4a12-aeaa-c810fb988a28 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:03.188855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504fe14-8d54-4a12-aeaa-c810fb988a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:03.189127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504fe14-8d54-4a12-aeaa-c810fb988a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:03.189755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504fe14-8d54-4a12-aeaa-c810fb988a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:03.192600 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504fe14-8d54-4a12-aeaa-c810fb988a28 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:03.230370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1011/2018] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:33:03 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:33:03.230972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6bbfb-d101-42ed-85c0-304ce21da8ac admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:03.242541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6bbfb-d101-42ed-85c0-304ce21da8ac admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:03.304194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ec67c-84f9-4066-b063-7d18d4c288bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:03.304833 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ec67c-84f9-4066-b063-7d18d4c288bd admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:03.306005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ec67c-84f9-4066-b063-7d18d4c288bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:03.306279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ec67c-84f9-4066-b063-7d18d4c288bd admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:03.306906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ec67c-84f9-4066-b063-7d18d4c288bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:03.308914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ec67c-84f9-4066-b063-7d18d4c288bd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:03.331328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1008/2019] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:33:03 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:33:03.335771 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504fe14-8d54-4a12-aeaa-c810fb988a28 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:03.347135 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504fe14-8d54-4a12-aeaa-c810fb988a28 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:03.351819 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9b6c0-9328-462c-9c98-32de45da4ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:03.352465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9b6c0-9328-462c-9c98-32de45da4ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:03.352763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9b6c0-9328-462c-9c98-32de45da4ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:03.353062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9b6c0-9328-462c-9c98-32de45da4ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:03.353712 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9b6c0-9328-462c-9c98-32de45da4ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:03.355323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9b6c0-9328-462c-9c98-32de45da4ff4 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:03.368932 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1012/2020] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:03.371572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ec67c-84f9-4066-b063-7d18d4c288bd admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:03.382262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ec67c-84f9-4066-b063-7d18d4c288bd admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:03.389040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7986f6-b18d-4d2e-9b63-d5534248aacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:03.389743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7986f6-b18d-4d2e-9b63-d5534248aacd admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:03.390022 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7986f6-b18d-4d2e-9b63-d5534248aacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:03.390293 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7986f6-b18d-4d2e-9b63-d5534248aacd admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:03.390937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7986f6-b18d-4d2e-9b63-d5534248aacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:03.392568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7986f6-b18d-4d2e-9b63-d5534248aacd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:03.404807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1009/2021] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:33:03.408657 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9b6c0-9328-462c-9c98-32de45da4ff4 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:03.419995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9b6c0-9328-462c-9c98-32de45da4ff4 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:03.424929 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de2376-569e-4f9b-a97e-abde2e1c9c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:03.425610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de2376-569e-4f9b-a97e-abde2e1c9c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:03.425888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de2376-569e-4f9b-a97e-abde2e1c9c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:03.426154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de2376-569e-4f9b-a97e-abde2e1c9c8e admin admin] PATH_INFO: `/v3/projects/7e514d27692f45ad9c5b451aa9c3837e/users/da5d71162e9f467bb873fa729eb79032/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:03.442355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de2376-569e-4f9b-a97e-abde2e1c9c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e514d27692f45ad9c5b451aa9c3837e, user_id=da5d71162e9f467bb873fa729eb79032, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:03.445249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de2376-569e-4f9b-a97e-abde2e1c9c8e admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:03.459844 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1013/2022] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:33:03 2022] PUT /identity/v3/projects/7e514d27692f45ad9c5b451aa9c3837e/users/da5d71162e9f467bb873fa729eb79032/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:03.471431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2c77d-e6b6-4e6e-9da2-ac233d268895 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:03.471722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2c77d-e6b6-4e6e-9da2-ac233d268895 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:03.471999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2c77d-e6b6-4e6e-9da2-ac233d268895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:03.505583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-93e2c77d-e6b6-4e6e-9da2-ac233d268895 None None] MFA Rules not processed for user `da5d71162e9f467bb873fa729eb79032`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:03.506865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-93e2c77d-e6b6-4e6e-9da2-ac233d268895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:03.574048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1010/2023] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:03.578668 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de2376-569e-4f9b-a97e-abde2e1c9c8e admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:03.615024 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:03.647343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de2376-569e-4f9b-a97e-abde2e1c9c8e admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:03.657291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38672c78-1cb9-4fb4-8d2f-7a1127888131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:03.657930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38672c78-1cb9-4fb4-8d2f-7a1127888131 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:03.658207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38672c78-1cb9-4fb4-8d2f-7a1127888131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:03.658481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38672c78-1cb9-4fb4-8d2f-7a1127888131 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:03.659178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38672c78-1cb9-4fb4-8d2f-7a1127888131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:03.661134 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38672c78-1cb9-4fb4-8d2f-7a1127888131 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:03.699981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1014/2024] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:33:03 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:33:03.701863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2c77d-e6b6-4e6e-9da2-ac233d268895 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:03.716473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2c77d-e6b6-4e6e-9da2-ac233d268895 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:03.765091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:03.817564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfa49f-3f1f-4950-8189-2944ee6e71c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:03.818196 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfa49f-3f1f-4950-8189-2944ee6e71c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:03.818520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfa49f-3f1f-4950-8189-2944ee6e71c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:03.818810 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfa49f-3f1f-4950-8189-2944ee6e71c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:03.819376 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfa49f-3f1f-4950-8189-2944ee6e71c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:03.821700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfa49f-3f1f-4950-8189-2944ee6e71c7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:03.858637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1011/2025] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:33:03 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:33:03.860126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38672c78-1cb9-4fb4-8d2f-7a1127888131 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:03.883327 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38672c78-1cb9-4fb4-8d2f-7a1127888131 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:03.883327 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48613005-be5c-4de2-8d80-38b9c7041d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:03.883327 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48613005-be5c-4de2-8d80-38b9c7041d92 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:03.883327 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48613005-be5c-4de2-8d80-38b9c7041d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:03.884294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48613005-be5c-4de2-8d80-38b9c7041d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:03.884294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48613005-be5c-4de2-8d80-38b9c7041d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:03.884294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48613005-be5c-4de2-8d80-38b9c7041d92 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:03.900792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1015/2026] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:03.903100 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfa49f-3f1f-4950-8189-2944ee6e71c7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:03.913672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfa49f-3f1f-4950-8189-2944ee6e71c7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:03.919387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced8daa-8bda-47fd-aafc-a1ceef4ee0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:03.920215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced8daa-8bda-47fd-aafc-a1ceef4ee0b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:03.920495 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced8daa-8bda-47fd-aafc-a1ceef4ee0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:03.920843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced8daa-8bda-47fd-aafc-a1ceef4ee0b2 admin admin] PATH_INFO: `/v3/projects/643c9c3a90824ac59030a15133240174/users/a3f6ce85c86643a18b4aa46058103296/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:03.938712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced8daa-8bda-47fd-aafc-a1ceef4ee0b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643c9c3a90824ac59030a15133240174, user_id=a3f6ce85c86643a18b4aa46058103296, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:03.942047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced8daa-8bda-47fd-aafc-a1ceef4ee0b2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:03.955509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1012/2027] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:33:03 2022] PUT /identity/v3/projects/643c9c3a90824ac59030a15133240174/users/a3f6ce85c86643a18b4aa46058103296/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:03.960031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48613005-be5c-4de2-8d80-38b9c7041d92 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:03.971597 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48613005-be5c-4de2-8d80-38b9c7041d92 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:04.028897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c922d0-c66e-496e-8b75-d9d59b829516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:04.029431 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c922d0-c66e-496e-8b75-d9d59b829516 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:04.029716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c922d0-c66e-496e-8b75-d9d59b829516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:04.030267 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c922d0-c66e-496e-8b75-d9d59b829516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:04.030618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c922d0-c66e-496e-8b75-d9d59b829516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:04.032207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c922d0-c66e-496e-8b75-d9d59b829516 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:04.044793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1016/2028] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:33:04.047150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced8daa-8bda-47fd-aafc-a1ceef4ee0b2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:04.058570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced8daa-8bda-47fd-aafc-a1ceef4ee0b2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:04.063300 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869540b-e8d0-4e54-8d66-25c35f8a0b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:04.063958 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869540b-e8d0-4e54-8d66-25c35f8a0b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:04.064243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869540b-e8d0-4e54-8d66-25c35f8a0b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:04.064520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869540b-e8d0-4e54-8d66-25c35f8a0b5d admin admin] PATH_INFO: `/v3/projects/643c9c3a90824ac59030a15133240174/users/a3f6ce85c86643a18b4aa46058103296/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:04.066726 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869540b-e8d0-4e54-8d66-25c35f8a0b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=643c9c3a90824ac59030a15133240174, user_id=a3f6ce85c86643a18b4aa46058103296, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:04.069172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869540b-e8d0-4e54-8d66-25c35f8a0b5d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:04.080658 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1013/2029] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:33:04 2022] PUT /identity/v3/projects/643c9c3a90824ac59030a15133240174/users/a3f6ce85c86643a18b4aa46058103296/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:33:04.088423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c922d0-c66e-496e-8b75-d9d59b829516 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:04.090197 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-94c922d0-c66e-496e-8b75-d9d59b829516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:04.090860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f57df4-6015-4009-b693-9350149e5b7f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:04.091150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f57df4-6015-4009-b693-9350149e5b7f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:04.091425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f57df4-6015-4009-b693-9350149e5b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:04.120546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-81f57df4-6015-4009-b693-9350149e5b7f None None] MFA Rules not processed for user `a3f6ce85c86643a18b4aa46058103296`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:04.121797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-81f57df4-6015-4009-b693-9350149e5b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:04.148774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c922d0-c66e-496e-8b75-d9d59b829516 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:04.194611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1014/2030] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:33:04.199212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51289df3-a66c-401d-997b-0f2923b32705 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:04.199433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51289df3-a66c-401d-997b-0f2923b32705 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:04.199828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51289df3-a66c-401d-997b-0f2923b32705 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:04.200106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51289df3-a66c-401d-997b-0f2923b32705 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:04.201894 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-51289df3-a66c-401d-997b-0f2923b32705 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:04.206627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f57df4-6015-4009-b693-9350149e5b7f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:04.208081 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-81f57df4-6015-4009-b693-9350149e5b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:04.215521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:04.220781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f57df4-6015-4009-b693-9350149e5b7f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:04.224643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a792b-4db5-4b94-984b-0a394a2575dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:04.225312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a792b-4db5-4b94-984b-0a394a2575dd service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:04.225597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a792b-4db5-4b94-984b-0a394a2575dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:04.225873 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a792b-4db5-4b94-984b-0a394a2575dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:04.269941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51289df3-a66c-401d-997b-0f2923b32705 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:04.271376 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51289df3-a66c-401d-997b-0f2923b32705 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:04.281884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1017/2031] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:04.286419 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a792b-4db5-4b94-984b-0a394a2575dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:04.287870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a792b-4db5-4b94-984b-0a394a2575dd service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:04.298690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1015/2032] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:33:06.868546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51289df3-a66c-401d-997b-0f2923b32705 service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:06.948518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51289df3-a66c-401d-997b-0f2923b32705 service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:06.954424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:06.955146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:06.955356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:06.955627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] PATH_INFO: `/v3/users/da5d71162e9f467bb873fa729eb79032` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:06.956888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5d71162e9f467bb873fa729eb79032)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:06.959245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:07.009996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5d71162e9f467bb873fa729eb79032'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.010682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5d71162e9f467bb873fa729eb79032'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.011291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5d71162e9f467bb873fa729eb79032'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.019964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5d71162e9f467bb873fa729eb79032'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.034037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5d71162e9f467bb873fa729eb79032'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.048172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1018/2033] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:33:06 2022] DELETE /identity/v3/users/da5d71162e9f467bb873fa729eb79032 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:33:07.052440 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9597034-c0a0-4d78-b6fb-a4afa31b7adf None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:07.052956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9597034-c0a0-4d78-b6fb-a4afa31b7adf None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:07.053047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9597034-c0a0-4d78-b6fb-a4afa31b7adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:07.072510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c9597034-c0a0-4d78-b6fb-a4afa31b7adf None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:07.073772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c9597034-c0a0-4d78-b6fb-a4afa31b7adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:07.077884 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:07.141523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1016/2034] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:33:07.151800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:07.153340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:07.189381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:07.217682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b738f92-3fa0-4eb5-8434-e5a055633c80 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:07.221296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ce1e2-40c7-4efb-92fc-f8f2acb46f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:07.221910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ce1e2-40c7-4efb-92fc-f8f2acb46f1c service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:07.222186 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ce1e2-40c7-4efb-92fc-f8f2acb46f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:07.222457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ce1e2-40c7-4efb-92fc-f8f2acb46f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:07.233905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ce1e2-40c7-4efb-92fc-f8f2acb46f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:07.235355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ce1e2-40c7-4efb-92fc-f8f2acb46f1c service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:07.245324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1019/2035] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:07.610068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9597034-c0a0-4d78-b6fb-a4afa31b7adf None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:07.612029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c9597034-c0a0-4d78-b6fb-a4afa31b7adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:07.626893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9597034-c0a0-4d78-b6fb-a4afa31b7adf None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:07.631872 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:07.632509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:07.633388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:07.633832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] PATH_INFO: `/v3/projects/7e514d27692f45ad9c5b451aa9c3837e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:07.634986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e514d27692f45ad9c5b451aa9c3837e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:07.636977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:07.700369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e514d27692f45ad9c5b451aa9c3837e', 'request_id': 'req-94889e28-7dca-4db4-9d3d-d137eafa7b5f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.705904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e514d27692f45ad9c5b451aa9c3837e', 'request_id': 'req-94889e28-7dca-4db4-9d3d-d137eafa7b5f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.712933 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e514d27692f45ad9c5b451aa9c3837e', 'request_id': 'req-94889e28-7dca-4db4-9d3d-d137eafa7b5f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.726050 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] The token cache is being invalidate because project 7e514d27692f45ad9c5b451aa9c3837e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:33:07.726338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.727549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.729488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1017/2036] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:33:07 2022] DELETE /identity/v3/projects/7e514d27692f45ad9c5b451aa9c3837e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:33:07.733694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ce1e2-40c7-4efb-92fc-f8f2acb46f1c service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:07.744527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ce1e2-40c7-4efb-92fc-f8f2acb46f1c service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:07.804733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7a0e5-23f1-4c11-af51-22e88ddb277c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:07.805418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7a0e5-23f1-4c11-af51-22e88ddb277c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:07.805698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7a0e5-23f1-4c11-af51-22e88ddb277c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:07.805970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7a0e5-23f1-4c11-af51-22e88ddb277c admin admin] PATH_INFO: `/v3/users/a3f6ce85c86643a18b4aa46058103296` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:07.807080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7a0e5-23f1-4c11-af51-22e88ddb277c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3f6ce85c86643a18b4aa46058103296)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:07.809020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7a0e5-23f1-4c11-af51-22e88ddb277c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:07.854020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-38d7a0e5-23f1-4c11-af51-22e88ddb277c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3f6ce85c86643a18b4aa46058103296'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.854663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-38d7a0e5-23f1-4c11-af51-22e88ddb277c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3f6ce85c86643a18b4aa46058103296'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.855259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-38d7a0e5-23f1-4c11-af51-22e88ddb277c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3f6ce85c86643a18b4aa46058103296'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.862081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-38d7a0e5-23f1-4c11-af51-22e88ddb277c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3f6ce85c86643a18b4aa46058103296'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.873878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-38d7a0e5-23f1-4c11-af51-22e88ddb277c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3f6ce85c86643a18b4aa46058103296'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:07.886081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1020/2037] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:33:07 2022] DELETE /identity/v3/users/a3f6ce85c86643a18b4aa46058103296 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:33:08.151696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:08.153671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:08.171910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94889e28-7dca-4db4-9d3d-d137eafa7b5f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:08.233669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348448-f2c4-46e4-8751-7f54245ceba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:08.234308 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348448-f2c4-46e4-8751-7f54245ceba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:08.234587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348448-f2c4-46e4-8751-7f54245ceba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:08.234862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348448-f2c4-46e4-8751-7f54245ceba0 admin admin] PATH_INFO: `/v3/projects/643c9c3a90824ac59030a15133240174` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:08.235986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87348448-f2c4-46e4-8751-7f54245ceba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=643c9c3a90824ac59030a15133240174)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:08.237927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87348448-f2c4-46e4-8751-7f54245ceba0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:08.276858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:08.288346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-87348448-f2c4-46e4-8751-7f54245ceba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643c9c3a90824ac59030a15133240174', 'request_id': 'req-87348448-f2c4-46e4-8751-7f54245ceba0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:08.294843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-87348448-f2c4-46e4-8751-7f54245ceba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643c9c3a90824ac59030a15133240174', 'request_id': 'req-87348448-f2c4-46e4-8751-7f54245ceba0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:08.301369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-87348448-f2c4-46e4-8751-7f54245ceba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643c9c3a90824ac59030a15133240174', 'request_id': 'req-87348448-f2c4-46e4-8751-7f54245ceba0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:08.313884 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-87348448-f2c4-46e4-8751-7f54245ceba0 admin admin] The token cache is being invalidate because project 643c9c3a90824ac59030a15133240174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:33:08.314181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-87348448-f2c4-46e4-8751-7f54245ceba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:08.314838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-87348448-f2c4-46e4-8751-7f54245ceba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:08.316493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1018/2038] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:33:08 2022] DELETE /identity/v3/projects/643c9c3a90824ac59030a15133240174 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:08.322504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22979dc-bdb1-4fc7-b4db-90e28bb1df5e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:08.322797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22979dc-bdb1-4fc7-b4db-90e28bb1df5e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:08.323082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22979dc-bdb1-4fc7-b4db-90e28bb1df5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:08.343831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-a22979dc-bdb1-4fc7-b4db-90e28bb1df5e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:08.345092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a22979dc-bdb1-4fc7-b4db-90e28bb1df5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:08.391376 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:08.408843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1021/2039] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:08.417898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90e6fb-d8b0-4a7a-b64e-396de2d089ac None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:08.418181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90e6fb-d8b0-4a7a-b64e-396de2d089ac None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:08.418459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90e6fb-d8b0-4a7a-b64e-396de2d089ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:08.449266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ad90e6fb-d8b0-4a7a-b64e-396de2d089ac None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:08.450924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ad90e6fb-d8b0-4a7a-b64e-396de2d089ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:08.478392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1019/2040] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:08.482727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22979dc-bdb1-4fc7-b4db-90e28bb1df5e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:08.505299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22979dc-bdb1-4fc7-b4db-90e28bb1df5e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:08.513113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700d1c9-5b4a-4b89-b312-4bbad2ecf611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:08.513897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700d1c9-5b4a-4b89-b312-4bbad2ecf611 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:08.514187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700d1c9-5b4a-4b89-b312-4bbad2ecf611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:08.514467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700d1c9-5b4a-4b89-b312-4bbad2ecf611 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:08.515528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700d1c9-5b4a-4b89-b312-4bbad2ecf611 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:33:08.516048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700d1c9-5b4a-4b89-b312-4bbad2ecf611 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:08.518085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700d1c9-5b4a-4b89-b312-4bbad2ecf611 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:08.533532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1022/2041] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:33:08.537772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90e6fb-d8b0-4a7a-b64e-396de2d089ac None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:08.549845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90e6fb-d8b0-4a7a-b64e-396de2d089ac None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:08.561702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15d4ee-7c79-4226-b669-09c18022486d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:08.562409 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15d4ee-7c79-4226-b669-09c18022486d admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:08.562750 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15d4ee-7c79-4226-b669-09c18022486d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:08.563082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15d4ee-7c79-4226-b669-09c18022486d admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:08.563800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15d4ee-7c79-4226-b669-09c18022486d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:08.566005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15d4ee-7c79-4226-b669-09c18022486d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:08.621885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1020/2042] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:33:08 2022] POST /identity/v3/projects => generated 391 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:33:08.622617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700d1c9-5b4a-4b89-b312-4bbad2ecf611 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:08.640509 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700d1c9-5b4a-4b89-b312-4bbad2ecf611 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:08.719042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3ca8a-a48a-477b-9cb4-7b814d7691b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:08.719654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3ca8a-a48a-477b-9cb4-7b814d7691b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:08.719937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3ca8a-a48a-477b-9cb4-7b814d7691b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:08.720213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3ca8a-a48a-477b-9cb4-7b814d7691b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:08.720879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3ca8a-a48a-477b-9cb4-7b814d7691b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:08.722784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3ca8a-a48a-477b-9cb4-7b814d7691b7 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:08.745995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1023/2043] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:33:08 2022] POST /identity/v3/users => generated 296 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:33:08.750043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15d4ee-7c79-4226-b669-09c18022486d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:08.760591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15d4ee-7c79-4226-b669-09c18022486d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:08.765295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca657d87-8dc7-4ba7-9f19-653b3aac1e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:08.765894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca657d87-8dc7-4ba7-9f19-653b3aac1e6f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:08.766177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca657d87-8dc7-4ba7-9f19-653b3aac1e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:08.766451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca657d87-8dc7-4ba7-9f19-653b3aac1e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:08.767093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca657d87-8dc7-4ba7-9f19-653b3aac1e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:08.768689 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca657d87-8dc7-4ba7-9f19-653b3aac1e6f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:08.779555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1021/2044] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:08.783384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3ca8a-a48a-477b-9cb4-7b814d7691b7 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:08.794930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3ca8a-a48a-477b-9cb4-7b814d7691b7 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:08.799612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d2cb2-e5cd-4a98-be70-bbf18bd41006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:08.800364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d2cb2-e5cd-4a98-be70-bbf18bd41006 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:08.800526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d2cb2-e5cd-4a98-be70-bbf18bd41006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:08.800829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d2cb2-e5cd-4a98-be70-bbf18bd41006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:08.801665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d2cb2-e5cd-4a98-be70-bbf18bd41006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:08.803304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d2cb2-e5cd-4a98-be70-bbf18bd41006 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:08.814571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1024/2045] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:08.818804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca657d87-8dc7-4ba7-9f19-653b3aac1e6f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:08.829077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca657d87-8dc7-4ba7-9f19-653b3aac1e6f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:08.833992 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618649dc-7a12-44c1-b579-559502530c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:08.834643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618649dc-7a12-44c1-b579-559502530c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:08.834925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618649dc-7a12-44c1-b579-559502530c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:08.835206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618649dc-7a12-44c1-b579-559502530c29 admin admin] PATH_INFO: `/v3/projects/0c2c01c3b9334ef1975088a201b9892c/users/aab7d767b0474e659485a2b911c2c9d1/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:08.850229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618649dc-7a12-44c1-b579-559502530c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c2c01c3b9334ef1975088a201b9892c, user_id=aab7d767b0474e659485a2b911c2c9d1, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:08.853466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618649dc-7a12-44c1-b579-559502530c29 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:08.866729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1022/2046] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:33:08 2022] PUT /identity/v3/projects/0c2c01c3b9334ef1975088a201b9892c/users/aab7d767b0474e659485a2b911c2c9d1/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:08.877146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14151973-1073-4fa7-a2e1-284e1a15bc96 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:08.877594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14151973-1073-4fa7-a2e1-284e1a15bc96 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:08.877763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14151973-1073-4fa7-a2e1-284e1a15bc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:08.905578 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-14151973-1073-4fa7-a2e1-284e1a15bc96 None None] MFA Rules not processed for user `aab7d767b0474e659485a2b911c2c9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:08.906829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-14151973-1073-4fa7-a2e1-284e1a15bc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:08.970160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1025/2047] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:08.980914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618649dc-7a12-44c1-b579-559502530c29 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:08.982673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-618649dc-7a12-44c1-b579-559502530c29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:09.043577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618649dc-7a12-44c1-b579-559502530c29 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:09.047555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d087f-9f63-4067-9fcf-eed5c272f6f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:09.048903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d087f-9f63-4067-9fcf-eed5c272f6f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:09.049195 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d087f-9f63-4067-9fcf-eed5c272f6f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:09.049469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d087f-9f63-4067-9fcf-eed5c272f6f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:09.058862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d087f-9f63-4067-9fcf-eed5c272f6f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:09.060392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d087f-9f63-4067-9fcf-eed5c272f6f8 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:09.069548 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1023/2048] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:18.967275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:19.065378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:27.631678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a10f5-d111-42ec-be1c-99eb2fb21948 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:27.632183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a10f5-d111-42ec-be1c-99eb2fb21948 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:27.632382 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a10f5-d111-42ec-be1c-99eb2fb21948 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:27.633846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1026/2049] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:33:27.650758 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ee7df-26f1-471d-867c-3c459124d7a3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:27.651731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ee7df-26f1-471d-867c-3c459124d7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:27.652076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ee7df-26f1-471d-867c-3c459124d7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:27.672840 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-264ee7df-26f1-471d-867c-3c459124d7a3 None None] MFA Rules not processed for user `1dcbe2b128a94b9abbf3a9b25db1e845`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:27.674094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-264ee7df-26f1-471d-867c-3c459124d7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:27.750905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1024/2050] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:33:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:27.764351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a10f5-d111-42ec-be1c-99eb2fb21948 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:27.766357 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-662a10f5-d111-42ec-be1c-99eb2fb21948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:27.784304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a10f5-d111-42ec-be1c-99eb2fb21948 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:27.790629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d071162-afff-4cd4-a731-5e9f50d6e404 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:27.791901 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d071162-afff-4cd4-a731-5e9f50d6e404 service placement] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:27.792388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d071162-afff-4cd4-a731-5e9f50d6e404 service placement] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:27.792967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d071162-afff-4cd4-a731-5e9f50d6e404 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:27.802723 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d071162-afff-4cd4-a731-5e9f50d6e404 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:27.805188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d071162-afff-4cd4-a731-5e9f50d6e404 service placement] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:27.815995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1027/2051] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:28.269260 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ee7df-26f1-471d-867c-3c459124d7a3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:28.270817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-264ee7df-26f1-471d-867c-3c459124d7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:28.340914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ee7df-26f1-471d-867c-3c459124d7a3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:28.346107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:28.346744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:28.347109 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:28.347467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] PATH_INFO: `/v3/users/ebb681992237466db91cc3df29c90f09` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:28.348619 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebb681992237466db91cc3df29c90f09)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:28.350763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:28.402599 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebb681992237466db91cc3df29c90f09'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.403256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebb681992237466db91cc3df29c90f09'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.403861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebb681992237466db91cc3df29c90f09'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.416125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebb681992237466db91cc3df29c90f09'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.423033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebb681992237466db91cc3df29c90f09'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.436740 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1025/2052] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:33:28 2022] DELETE /identity/v3/users/ebb681992237466db91cc3df29c90f09 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:28.611953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d071162-afff-4cd4-a731-5e9f50d6e404 service placement] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:28.627699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d071162-afff-4cd4-a731-5e9f50d6e404 service placement] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:28.702849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:28.703533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:28.703738 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:28.704008 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] PATH_INFO: `/v3/projects/07a31333a2f14aeca6cc20739c72bbc3` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:28.705153 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07a31333a2f14aeca6cc20739c72bbc3)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:28.707116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:28.752233 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07a31333a2f14aeca6cc20739c72bbc3', 'request_id': 'req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.758334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07a31333a2f14aeca6cc20739c72bbc3', 'request_id': 'req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.763782 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07a31333a2f14aeca6cc20739c72bbc3', 'request_id': 'req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.776634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] The token cache is being invalidate because project 07a31333a2f14aeca6cc20739c72bbc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:33:28.776980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.777602 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.779115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1028/2053] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:33:28 2022] DELETE /identity/v3/projects/07a31333a2f14aeca6cc20739c72bbc3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:28.783000 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:28.784487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:28.798436 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e385aa-97ca-40cd-a14f-b6145c8e31cb admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:28.811073 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:28.860994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec911b-4a35-49b3-9e47-2d33bc8a532e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:28.861555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec911b-4a35-49b3-9e47-2d33bc8a532e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:28.861834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec911b-4a35-49b3-9e47-2d33bc8a532e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:28.862107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec911b-4a35-49b3-9e47-2d33bc8a532e admin admin] PATH_INFO: `/v3/users/5da568a7163b4535a4dca6fbe038bea3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:28.863222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec911b-4a35-49b3-9e47-2d33bc8a532e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da568a7163b4535a4dca6fbe038bea3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:28.865185 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec911b-4a35-49b3-9e47-2d33bc8a532e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:28.918697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-24ec911b-4a35-49b3-9e47-2d33bc8a532e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da568a7163b4535a4dca6fbe038bea3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.919349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-24ec911b-4a35-49b3-9e47-2d33bc8a532e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da568a7163b4535a4dca6fbe038bea3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.919981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-24ec911b-4a35-49b3-9e47-2d33bc8a532e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da568a7163b4535a4dca6fbe038bea3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.932965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-24ec911b-4a35-49b3-9e47-2d33bc8a532e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da568a7163b4535a4dca6fbe038bea3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.942155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-24ec911b-4a35-49b3-9e47-2d33bc8a532e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da568a7163b4535a4dca6fbe038bea3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:28.966771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1026/2054] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:33:28 2022] DELETE /identity/v3/users/5da568a7163b4535a4dca6fbe038bea3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:29.152281 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:29.153746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:29.168348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce727e84-49e1-4a88-a8ee-e31c202bbcc8 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:29.224922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112076f4-f0a8-478b-8a23-edc89d6b6b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:29.226242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112076f4-f0a8-478b-8a23-edc89d6b6b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:29.226526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112076f4-f0a8-478b-8a23-edc89d6b6b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:29.226798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112076f4-f0a8-478b-8a23-edc89d6b6b7b admin admin] PATH_INFO: `/v3/projects/de32071c5b2346ff85920301f838d296` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:29.227928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112076f4-f0a8-478b-8a23-edc89d6b6b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=de32071c5b2346ff85920301f838d296)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:29.229875 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112076f4-f0a8-478b-8a23-edc89d6b6b7b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:29.251498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:29.278150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-112076f4-f0a8-478b-8a23-edc89d6b6b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de32071c5b2346ff85920301f838d296', 'request_id': 'req-112076f4-f0a8-478b-8a23-edc89d6b6b7b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:29.284204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-112076f4-f0a8-478b-8a23-edc89d6b6b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de32071c5b2346ff85920301f838d296', 'request_id': 'req-112076f4-f0a8-478b-8a23-edc89d6b6b7b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:29.290170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-112076f4-f0a8-478b-8a23-edc89d6b6b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de32071c5b2346ff85920301f838d296', 'request_id': 'req-112076f4-f0a8-478b-8a23-edc89d6b6b7b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:29.302964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-112076f4-f0a8-478b-8a23-edc89d6b6b7b admin admin] The token cache is being invalidate because project de32071c5b2346ff85920301f838d296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:33:29.303271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-112076f4-f0a8-478b-8a23-edc89d6b6b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:29.304581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-112076f4-f0a8-478b-8a23-edc89d6b6b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:29.306318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1029/2055] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:33:29 2022] DELETE /identity/v3/projects/de32071c5b2346ff85920301f838d296 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:29.311211 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc0fcf-aad3-4c63-b344-b099855169c5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:29.311569 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc0fcf-aad3-4c63-b344-b099855169c5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:29.311921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc0fcf-aad3-4c63-b344-b099855169c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:29.331720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8cbc0fcf-aad3-4c63-b344-b099855169c5 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:29.333316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8cbc0fcf-aad3-4c63-b344-b099855169c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:29.392269 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1027/2056] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:33:29.400689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a1443-506f-4871-8e33-699af0ee96e1 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:29.401013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a1443-506f-4871-8e33-699af0ee96e1 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:29.401319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a1443-506f-4871-8e33-699af0ee96e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:29.422534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-2a8a1443-506f-4871-8e33-699af0ee96e1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:29.423747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2a8a1443-506f-4871-8e33-699af0ee96e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:29.433023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1030/2057] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:29.437290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc0fcf-aad3-4c63-b344-b099855169c5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:29.447884 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc0fcf-aad3-4c63-b344-b099855169c5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:29.453452 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23911ea8-984d-482f-9cee-3d0f321599bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:29.454074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23911ea8-984d-482f-9cee-3d0f321599bb admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:29.454365 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23911ea8-984d-482f-9cee-3d0f321599bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:29.454638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23911ea8-984d-482f-9cee-3d0f321599bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:29.455560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23911ea8-984d-482f-9cee-3d0f321599bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:33:29.455865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23911ea8-984d-482f-9cee-3d0f321599bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:29.457490 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23911ea8-984d-482f-9cee-3d0f321599bb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:29.473738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1028/2058] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:33:29.474990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a1443-506f-4871-8e33-699af0ee96e1 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:29.485236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a1443-506f-4871-8e33-699af0ee96e1 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:29.490254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645871c-c8ea-4103-8f70-fd9e1645fd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:29.493511 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645871c-c8ea-4103-8f70-fd9e1645fd27 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:29.493511 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645871c-c8ea-4103-8f70-fd9e1645fd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:29.493511 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645871c-c8ea-4103-8f70-fd9e1645fd27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:29.493511 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645871c-c8ea-4103-8f70-fd9e1645fd27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:29.494279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645871c-c8ea-4103-8f70-fd9e1645fd27 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:29.526682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1031/2059] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:33:29 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:33:29.527964 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23911ea8-984d-482f-9cee-3d0f321599bb admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:29.543464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23911ea8-984d-482f-9cee-3d0f321599bb admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:29.564040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:29.607915 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b6645-5190-4005-aaa1-08f9798a9c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:29.608648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b6645-5190-4005-aaa1-08f9798a9c3c admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:29.609026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b6645-5190-4005-aaa1-08f9798a9c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:29.609379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b6645-5190-4005-aaa1-08f9798a9c3c admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:29.610085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b6645-5190-4005-aaa1-08f9798a9c3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:29.612089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b6645-5190-4005-aaa1-08f9798a9c3c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:29.645014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1029/2060] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:33:29 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:33:29.645217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645871c-c8ea-4103-8f70-fd9e1645fd27 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:29.656189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645871c-c8ea-4103-8f70-fd9e1645fd27 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:29.662242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75582a-e0d5-416a-9a47-c91873a68571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:29.662969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75582a-e0d5-416a-9a47-c91873a68571 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:29.664968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75582a-e0d5-416a-9a47-c91873a68571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:29.664968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75582a-e0d5-416a-9a47-c91873a68571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:29.664968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75582a-e0d5-416a-9a47-c91873a68571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:29.666167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75582a-e0d5-416a-9a47-c91873a68571 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:29.679638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1032/2061] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:33:29.681806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b6645-5190-4005-aaa1-08f9798a9c3c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:29.692405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b6645-5190-4005-aaa1-08f9798a9c3c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:29.701332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48d1ed-3363-4b73-9ea9-e9389a328368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:29.701952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48d1ed-3363-4b73-9ea9-e9389a328368 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:29.702230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48d1ed-3363-4b73-9ea9-e9389a328368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:29.702501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48d1ed-3363-4b73-9ea9-e9389a328368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:29.703148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48d1ed-3363-4b73-9ea9-e9389a328368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:29.705073 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48d1ed-3363-4b73-9ea9-e9389a328368 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:29.716910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1030/2062] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:29.722418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75582a-e0d5-416a-9a47-c91873a68571 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:29.739756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75582a-e0d5-416a-9a47-c91873a68571 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:29.746389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b59762-fa3a-423e-b093-ab5faf85b68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:29.747043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b59762-fa3a-423e-b093-ab5faf85b68f admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:29.747324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b59762-fa3a-423e-b093-ab5faf85b68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:29.747597 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b59762-fa3a-423e-b093-ab5faf85b68f admin admin] PATH_INFO: `/v3/projects/5fe6b6f9e22741ce92e106199ef568b2/users/3d7d7cf8bcd4451ea0e540cfb3ab98c6/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:29.764686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b59762-fa3a-423e-b093-ab5faf85b68f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fe6b6f9e22741ce92e106199ef568b2, user_id=3d7d7cf8bcd4451ea0e540cfb3ab98c6, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:29.767812 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b59762-fa3a-423e-b093-ab5faf85b68f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:29.782013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1033/2063] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:33:29 2022] PUT /identity/v3/projects/5fe6b6f9e22741ce92e106199ef568b2/users/3d7d7cf8bcd4451ea0e540cfb3ab98c6/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:29.795201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e35de-9ecd-436b-986a-b5faebe5b541 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:29.795782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e35de-9ecd-436b-986a-b5faebe5b541 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:29.795782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e35de-9ecd-436b-986a-b5faebe5b541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:29.843509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-293e35de-9ecd-436b-986a-b5faebe5b541 None None] MFA Rules not processed for user `3d7d7cf8bcd4451ea0e540cfb3ab98c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:29.844818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-293e35de-9ecd-436b-986a-b5faebe5b541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:29.916867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1031/2064] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:29.932386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b59762-fa3a-423e-b093-ab5faf85b68f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:29.934481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-05b59762-fa3a-423e-b093-ab5faf85b68f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:29.999034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b59762-fa3a-423e-b093-ab5faf85b68f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:30.003646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d2567-a4d6-4c1e-be01-71edd2b98022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:30.004291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d2567-a4d6-4c1e-be01-71edd2b98022 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:30.004692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d2567-a4d6-4c1e-be01-71edd2b98022 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:30.004996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d2567-a4d6-4c1e-be01-71edd2b98022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:30.015199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d2567-a4d6-4c1e-be01-71edd2b98022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:30.016659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d2567-a4d6-4c1e-be01-71edd2b98022 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:30.034739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1034/2065] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:30.064778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e35de-9ecd-436b-986a-b5faebe5b541 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:30.141728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e35de-9ecd-436b-986a-b5faebe5b541 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:30.146330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:30.146887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:30.147220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:30.148089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] PATH_INFO: `/v3/users/3d7d7cf8bcd4451ea0e540cfb3ab98c6` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:30.149566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7d7cf8bcd4451ea0e540cfb3ab98c6)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:30.151612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:30.200908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:30.217329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7d7cf8bcd4451ea0e540cfb3ab98c6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:30.217982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7d7cf8bcd4451ea0e540cfb3ab98c6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:30.218593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7d7cf8bcd4451ea0e540cfb3ab98c6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:30.237254 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7d7cf8bcd4451ea0e540cfb3ab98c6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:30.239949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7d7cf8bcd4451ea0e540cfb3ab98c6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:30.253324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1032/2066] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:33:30 2022] DELETE /identity/v3/users/3d7d7cf8bcd4451ea0e540cfb3ab98c6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:30.258453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420564d2-8fb3-4a8d-8601-11e14a2ae4ab None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:30.258736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420564d2-8fb3-4a8d-8601-11e14a2ae4ab None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:30.259034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420564d2-8fb3-4a8d-8601-11e14a2ae4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:30.279243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-420564d2-8fb3-4a8d-8601-11e14a2ae4ab None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:30.280466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-420564d2-8fb3-4a8d-8601-11e14a2ae4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:30.328266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:30.348206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1035/2067] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:30.358611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:30.360130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:30.424222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512da12-0fdd-438f-aba9-91d8982aed02 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:30.430418 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0e782-612a-40fd-a33a-eca7b1e8ecb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:30.431130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0e782-612a-40fd-a33a-eca7b1e8ecb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:30.431583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0e782-612a-40fd-a33a-eca7b1e8ecb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:30.431933 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0e782-612a-40fd-a33a-eca7b1e8ecb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:30.442931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0e782-612a-40fd-a33a-eca7b1e8ecb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:30.445235 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0e782-612a-40fd-a33a-eca7b1e8ecb4 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:30.456275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1033/2068] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:30.749567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420564d2-8fb3-4a8d-8601-11e14a2ae4ab None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:30.750958 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-420564d2-8fb3-4a8d-8601-11e14a2ae4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:30.765591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420564d2-8fb3-4a8d-8601-11e14a2ae4ab None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:30.770423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60983225-20a7-4889-97e6-503d96de7d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:30.771545 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60983225-20a7-4889-97e6-503d96de7d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:30.771999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60983225-20a7-4889-97e6-503d96de7d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:30.772470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60983225-20a7-4889-97e6-503d96de7d7a admin admin] PATH_INFO: `/v3/projects/5fe6b6f9e22741ce92e106199ef568b2` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:30.773742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60983225-20a7-4889-97e6-503d96de7d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fe6b6f9e22741ce92e106199ef568b2)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:30.775604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60983225-20a7-4889-97e6-503d96de7d7a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:30.823751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-60983225-20a7-4889-97e6-503d96de7d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fe6b6f9e22741ce92e106199ef568b2', 'request_id': 'req-60983225-20a7-4889-97e6-503d96de7d7a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:30.830517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-60983225-20a7-4889-97e6-503d96de7d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fe6b6f9e22741ce92e106199ef568b2', 'request_id': 'req-60983225-20a7-4889-97e6-503d96de7d7a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:30.836782 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-60983225-20a7-4889-97e6-503d96de7d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fe6b6f9e22741ce92e106199ef568b2', 'request_id': 'req-60983225-20a7-4889-97e6-503d96de7d7a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:30.848538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-60983225-20a7-4889-97e6-503d96de7d7a admin admin] The token cache is being invalidate because project 5fe6b6f9e22741ce92e106199ef568b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:33:30.848854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-60983225-20a7-4889-97e6-503d96de7d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:30.849498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-60983225-20a7-4889-97e6-503d96de7d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:30.851774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1036/2069] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:33:30 2022] DELETE /identity/v3/projects/5fe6b6f9e22741ce92e106199ef568b2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:30.856776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f96b7-8d5a-4810-ad03-4b57bf4b8e97 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:30.857224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f96b7-8d5a-4810-ad03-4b57bf4b8e97 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:30.857347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f96b7-8d5a-4810-ad03-4b57bf4b8e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:30.878776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-764f96b7-8d5a-4810-ad03-4b57bf4b8e97 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:30.879980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-764f96b7-8d5a-4810-ad03-4b57bf4b8e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:30.941683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1034/2070] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:30.950138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ed42e-50af-474f-92b5-35db1a031ad3 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:30.950357 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ed42e-50af-474f-92b5-35db1a031ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:30.950819 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ed42e-50af-474f-92b5-35db1a031ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:30.973746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-346ed42e-50af-474f-92b5-35db1a031ad3 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:30.974961 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-346ed42e-50af-474f-92b5-35db1a031ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:30.984579 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1037/2071] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:33:30.988744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f96b7-8d5a-4810-ad03-4b57bf4b8e97 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:30.999592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f96b7-8d5a-4810-ad03-4b57bf4b8e97 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:31.006850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11f201-00a2-408b-b867-ee22694b0225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:31.007623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11f201-00a2-408b-b867-ee22694b0225 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:31.007930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11f201-00a2-408b-b867-ee22694b0225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:31.008289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11f201-00a2-408b-b867-ee22694b0225 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:31.009213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11f201-00a2-408b-b867-ee22694b0225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:33:31.009523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11f201-00a2-408b-b867-ee22694b0225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:31.011289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11f201-00a2-408b-b867-ee22694b0225 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:31.026570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1035/2072] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:33:31.030674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ed42e-50af-474f-92b5-35db1a031ad3 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:31.044887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ed42e-50af-474f-92b5-35db1a031ad3 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:31.049695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc01f3-6fc8-4a7c-8d08-64b624a88e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:31.050301 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc01f3-6fc8-4a7c-8d08-64b624a88e72 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:31.050586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc01f3-6fc8-4a7c-8d08-64b624a88e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:31.050857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc01f3-6fc8-4a7c-8d08-64b624a88e72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:31.051494 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc01f3-6fc8-4a7c-8d08-64b624a88e72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:31.053696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc01f3-6fc8-4a7c-8d08-64b624a88e72 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:31.087318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1038/2073] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:33:31 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:33:31.091479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11f201-00a2-408b-b867-ee22694b0225 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:31.104151 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11f201-00a2-408b-b867-ee22694b0225 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:31.153041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:31.180923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac196f-c097-43c4-844b-5dddd31d71dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:31.181541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac196f-c097-43c4-844b-5dddd31d71dd admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:31.181822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac196f-c097-43c4-844b-5dddd31d71dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:31.182093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac196f-c097-43c4-844b-5dddd31d71dd admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:31.182716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac196f-c097-43c4-844b-5dddd31d71dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:31.184665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac196f-c097-43c4-844b-5dddd31d71dd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:31.206656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1036/2074] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:33:31 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:33:31.214006 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc01f3-6fc8-4a7c-8d08-64b624a88e72 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:31.224682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc01f3-6fc8-4a7c-8d08-64b624a88e72 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:31.230217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35915b41-a48b-4be5-a0ab-fb44d47a5298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:31.230823 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35915b41-a48b-4be5-a0ab-fb44d47a5298 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:31.231106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35915b41-a48b-4be5-a0ab-fb44d47a5298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:31.231386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35915b41-a48b-4be5-a0ab-fb44d47a5298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:31.232030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35915b41-a48b-4be5-a0ab-fb44d47a5298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:31.233654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35915b41-a48b-4be5-a0ab-fb44d47a5298 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:31.245839 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1039/2075] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:31.249966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac196f-c097-43c4-844b-5dddd31d71dd admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:31.264537 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac196f-c097-43c4-844b-5dddd31d71dd admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:31.270344 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b0932-37a1-4f32-9e9d-e053874c742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:31.270962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b0932-37a1-4f32-9e9d-e053874c742f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:31.271244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b0932-37a1-4f32-9e9d-e053874c742f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:31.271520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b0932-37a1-4f32-9e9d-e053874c742f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:31.272715 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b0932-37a1-4f32-9e9d-e053874c742f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:31.274356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b0932-37a1-4f32-9e9d-e053874c742f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:31.292802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1037/2076] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:33:31.295731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c1aa9-5e45-443e-8687-d663cd2fdcee None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:31.296019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c1aa9-5e45-443e-8687-d663cd2fdcee None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:31.296127 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b0932-37a1-4f32-9e9d-e053874c742f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:31.296296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c1aa9-5e45-443e-8687-d663cd2fdcee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:31.306723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b0932-37a1-4f32-9e9d-e053874c742f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:31.312218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18c5a6-31ec-41f3-9f34-8cfbe1618232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:31.312900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18c5a6-31ec-41f3-9f34-8cfbe1618232 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:31.313178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18c5a6-31ec-41f3-9f34-8cfbe1618232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:31.313451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18c5a6-31ec-41f3-9f34-8cfbe1618232 admin admin] PATH_INFO: `/v3/projects/b34c07333ddf4a19b35424d47d19dc60/users/d9e505bb350e4134976000a83bdd4fe1/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:31.318620 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-749c1aa9-5e45-443e-8687-d663cd2fdcee None None] MFA Rules not processed for user `7536a92033384383bd625b6c969ebd38`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:31.321583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-749c1aa9-5e45-443e-8687-d663cd2fdcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:31.335682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18c5a6-31ec-41f3-9f34-8cfbe1618232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34c07333ddf4a19b35424d47d19dc60, user_id=d9e505bb350e4134976000a83bdd4fe1, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:31.338590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18c5a6-31ec-41f3-9f34-8cfbe1618232 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:31.357890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1038/2077] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:33:31 2022] PUT /identity/v3/projects/b34c07333ddf4a19b35424d47d19dc60/users/d9e505bb350e4134976000a83bdd4fe1/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:31.362545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c474ff5-d0d1-43b5-93dc-b38e0369324e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:31.362950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c474ff5-d0d1-43b5-93dc-b38e0369324e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:31.363182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c474ff5-d0d1-43b5-93dc-b38e0369324e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:31.384182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2c474ff5-d0d1-43b5-93dc-b38e0369324e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:31.385316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2c474ff5-d0d1-43b5-93dc-b38e0369324e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:31.403974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1040/2078] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:33:31.455769 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1039/2079] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:33:31.469379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c1aa9-5e45-443e-8687-d663cd2fdcee None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:31.470926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-749c1aa9-5e45-443e-8687-d663cd2fdcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:31.535884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:31.556976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c1aa9-5e45-443e-8687-d663cd2fdcee None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:31.609372 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186334a8-196d-471a-92de-66d4c992a962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:31.609984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186334a8-196d-471a-92de-66d4c992a962 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:31.610262 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186334a8-196d-471a-92de-66d4c992a962 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:31.611016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186334a8-196d-471a-92de-66d4c992a962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:31.623083 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186334a8-196d-471a-92de-66d4c992a962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:31.624527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186334a8-196d-471a-92de-66d4c992a962 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:31.634822 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1041/2080] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:31.670045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c474ff5-d0d1-43b5-93dc-b38e0369324e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:31.672269 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2c474ff5-d0d1-43b5-93dc-b38e0369324e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:31.686686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c474ff5-d0d1-43b5-93dc-b38e0369324e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:31.692002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:31.692972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:31.693445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:31.693907 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] PATH_INFO: `/v3/users/aab7d767b0474e659485a2b911c2c9d1` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:31.695474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab7d767b0474e659485a2b911c2c9d1)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:31.698103 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:31.758597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab7d767b0474e659485a2b911c2c9d1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:31.759379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab7d767b0474e659485a2b911c2c9d1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:31.760088 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab7d767b0474e659485a2b911c2c9d1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:31.775061 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab7d767b0474e659485a2b911c2c9d1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:31.786136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab7d767b0474e659485a2b911c2c9d1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:31.805430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1040/2081] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:33:31 2022] DELETE /identity/v3/users/aab7d767b0474e659485a2b911c2c9d1 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:33:31.810045 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20433d03-4286-4a46-ab21-d7864a869e65 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:31.811173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20433d03-4286-4a46-ab21-d7864a869e65 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:31.811173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20433d03-4286-4a46-ab21-d7864a869e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:31.853931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-20433d03-4286-4a46-ab21-d7864a869e65 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:31.855203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-20433d03-4286-4a46-ab21-d7864a869e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:31.937563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1042/2082] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:31.949172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:31.950850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:31.963014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:32.064249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fa09a-8e53-4f8c-8af7-9764a0b84b7e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:32.072573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de120761-914c-45ee-8567-ba03147ad62b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:32.073280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de120761-914c-45ee-8567-ba03147ad62b service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:32.073569 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de120761-914c-45ee-8567-ba03147ad62b service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:32.074020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de120761-914c-45ee-8567-ba03147ad62b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:32.089808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de120761-914c-45ee-8567-ba03147ad62b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:32.091374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de120761-914c-45ee-8567-ba03147ad62b service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:32.109319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1041/2083] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:32.818044 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9740e40-492d-433c-bd00-b74eee12d4e8 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:32.818650 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9740e40-492d-433c-bd00-b74eee12d4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:32.818765 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9740e40-492d-433c-bd00-b74eee12d4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:32.862838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de120761-914c-45ee-8567-ba03147ad62b service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:32.864777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-de120761-914c-45ee-8567-ba03147ad62b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:32.883164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-e9740e40-492d-433c-bd00-b74eee12d4e8 None None] MFA Rules not processed for user `d9e505bb350e4134976000a83bdd4fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:32.884677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e9740e40-492d-433c-bd00-b74eee12d4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:32.887838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de120761-914c-45ee-8567-ba03147ad62b service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:32.894747 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:32.895406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:32.895686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:32.895958 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd admin admin] PATH_INFO: `/v3/projects/0c2c01c3b9334ef1975088a201b9892c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:32.898093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c2c01c3b9334ef1975088a201b9892c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:32.900805 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:32.971337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c2c01c3b9334ef1975088a201b9892c', 'request_id': 'req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:32.979542 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c2c01c3b9334ef1975088a201b9892c', 'request_id': 'req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:32.989172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c2c01c3b9334ef1975088a201b9892c', 'request_id': 'req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:32.999104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1043/2084] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:33.006179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd admin admin] The token cache is being invalidate because project 0c2c01c3b9334ef1975088a201b9892c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:33:33.007333 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:33.008167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-12d93551-2a08-45b8-8ac7-2a80a9bd72cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:33.010776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1042/2085] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:33:32 2022] DELETE /identity/v3/projects/0c2c01c3b9334ef1975088a201b9892c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:33.018502 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9740e40-492d-433c-bd00-b74eee12d4e8 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:33.023879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e9740e40-492d-433c-bd00-b74eee12d4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:33.059376 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:33.097591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9740e40-492d-433c-bd00-b74eee12d4e8 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:33.166924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f66e4-0fbb-48af-ad81-ebe418dc807a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:33.167963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f66e4-0fbb-48af-ad81-ebe418dc807a service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:33.168212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f66e4-0fbb-48af-ad81-ebe418dc807a service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:33.168504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f66e4-0fbb-48af-ad81-ebe418dc807a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:33.171238 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-577f66e4-0fbb-48af-ad81-ebe418dc807a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:33.240501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f66e4-0fbb-48af-ad81-ebe418dc807a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:33.242166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f66e4-0fbb-48af-ad81-ebe418dc807a service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:33.254539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1044/2086] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:34.686407 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5460c7f-7e40-4be3-a7ed-604e0df1f67d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:34.688387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5460c7f-7e40-4be3-a7ed-604e0df1f67d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:34.688861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5460c7f-7e40-4be3-a7ed-604e0df1f67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:34.721774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d5460c7f-7e40-4be3-a7ed-604e0df1f67d None None] MFA Rules not processed for user `d9e505bb350e4134976000a83bdd4fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:34.723185 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d5460c7f-7e40-4be3-a7ed-604e0df1f67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:34.750885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1043/2087] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:37.368555 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f66e4-0fbb-48af-ad81-ebe418dc807a service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:37.370021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-577f66e4-0fbb-48af-ad81-ebe418dc807a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:37.440389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f66e4-0fbb-48af-ad81-ebe418dc807a service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:37.445517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ef0dc-e49a-45aa-9d06-b064f9bd5cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:37.446148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ef0dc-e49a-45aa-9d06-b064f9bd5cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:37.446430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ef0dc-e49a-45aa-9d06-b064f9bd5cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:37.446709 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ef0dc-e49a-45aa-9d06-b064f9bd5cad admin admin] PATH_INFO: `/v3/users/8d20b306fc68435baa0da6f09c915157` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:37.447870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ef0dc-e49a-45aa-9d06-b064f9bd5cad admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d20b306fc68435baa0da6f09c915157)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:37.449837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ef0dc-e49a-45aa-9d06-b064f9bd5cad admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:37.494826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:37.506087 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cd5ef0dc-e49a-45aa-9d06-b064f9bd5cad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d20b306fc68435baa0da6f09c915157'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:37.507434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cd5ef0dc-e49a-45aa-9d06-b064f9bd5cad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d20b306fc68435baa0da6f09c915157'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:37.508878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cd5ef0dc-e49a-45aa-9d06-b064f9bd5cad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d20b306fc68435baa0da6f09c915157'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:37.518387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cd5ef0dc-e49a-45aa-9d06-b064f9bd5cad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d20b306fc68435baa0da6f09c915157'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:37.535712 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cd5ef0dc-e49a-45aa-9d06-b064f9bd5cad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d20b306fc68435baa0da6f09c915157'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:37.556414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1045/2088] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:33:37 2022] DELETE /identity/v3/users/8d20b306fc68435baa0da6f09c915157 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:37.964203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5460c7f-7e40-4be3-a7ed-604e0df1f67d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:37.977171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5460c7f-7e40-4be3-a7ed-604e0df1f67d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:38.051769 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99170d-86ae-4979-9caf-721d431c6ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:38.052412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99170d-86ae-4979-9caf-721d431c6ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.052886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99170d-86ae-4979-9caf-721d431c6ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.053772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99170d-86ae-4979-9caf-721d431c6ad2 admin admin] PATH_INFO: `/v3/projects/b4516d3cd63d4d89824c4a0b20b6eea1` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.054898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99170d-86ae-4979-9caf-721d431c6ad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4516d3cd63d4d89824c4a0b20b6eea1)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:38.056874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99170d-86ae-4979-9caf-721d431c6ad2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:38.089000 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:38.105519 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-db99170d-86ae-4979-9caf-721d431c6ad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4516d3cd63d4d89824c4a0b20b6eea1', 'request_id': 'req-db99170d-86ae-4979-9caf-721d431c6ad2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:38.110940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-db99170d-86ae-4979-9caf-721d431c6ad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4516d3cd63d4d89824c4a0b20b6eea1', 'request_id': 'req-db99170d-86ae-4979-9caf-721d431c6ad2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:38.118139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-db99170d-86ae-4979-9caf-721d431c6ad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4516d3cd63d4d89824c4a0b20b6eea1', 'request_id': 'req-db99170d-86ae-4979-9caf-721d431c6ad2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:38.130345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-db99170d-86ae-4979-9caf-721d431c6ad2 admin admin] The token cache is being invalidate because project b4516d3cd63d4d89824c4a0b20b6eea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:33:38.130643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-db99170d-86ae-4979-9caf-721d431c6ad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:38.131975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-db99170d-86ae-4979-9caf-721d431c6ad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:38.133683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1044/2089] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:33:37 2022] DELETE /identity/v3/projects/b4516d3cd63d4d89824c4a0b20b6eea1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:38.138558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e210933-0c8c-42fa-899c-2249aebb5d3e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.138844 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e210933-0c8c-42fa-899c-2249aebb5d3e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.139118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e210933-0c8c-42fa-899c-2249aebb5d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.165506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3e210933-0c8c-42fa-899c-2249aebb5d3e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:38.166766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3e210933-0c8c-42fa-899c-2249aebb5d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:38.266596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1046/2090] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:38.274844 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84789a7f-47b8-453f-a0fc-866a0ce781e9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.275134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84789a7f-47b8-453f-a0fc-866a0ce781e9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.275514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84789a7f-47b8-453f-a0fc-866a0ce781e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.297194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-84789a7f-47b8-453f-a0fc-866a0ce781e9 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:38.298420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-84789a7f-47b8-453f-a0fc-866a0ce781e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:38.307169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1045/2091] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:33:38.311571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e210933-0c8c-42fa-899c-2249aebb5d3e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:38.321810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e210933-0c8c-42fa-899c-2249aebb5d3e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:38.326566 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736b891-a942-4e91-ab2d-2e5ac846be0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:38.327239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736b891-a942-4e91-ab2d-2e5ac846be0b admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.327609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736b891-a942-4e91-ab2d-2e5ac846be0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.327887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736b891-a942-4e91-ab2d-2e5ac846be0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.328533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736b891-a942-4e91-ab2d-2e5ac846be0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:33:38.328861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736b891-a942-4e91-ab2d-2e5ac846be0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:38.330536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736b891-a942-4e91-ab2d-2e5ac846be0b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:38.346881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1047/2092] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:33:38.347760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84789a7f-47b8-453f-a0fc-866a0ce781e9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:38.357664 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84789a7f-47b8-453f-a0fc-866a0ce781e9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:38.362510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2789932-c445-4cbf-b1e0-b3933f585a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:38.363114 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2789932-c445-4cbf-b1e0-b3933f585a62 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.363392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2789932-c445-4cbf-b1e0-b3933f585a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.363665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2789932-c445-4cbf-b1e0-b3933f585a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.364295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2789932-c445-4cbf-b1e0-b3933f585a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:38.367233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2789932-c445-4cbf-b1e0-b3933f585a62 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:38.398596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1046/2093] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:33:38 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:33:38.399849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736b891-a942-4e91-ab2d-2e5ac846be0b admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:38.410735 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736b891-a942-4e91-ab2d-2e5ac846be0b admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:38.474582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7e765-f8a2-4cd3-8866-0234b33119e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:38.475347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7e765-f8a2-4cd3-8866-0234b33119e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.475687 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7e765-f8a2-4cd3-8866-0234b33119e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.476044 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7e765-f8a2-4cd3-8866-0234b33119e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.476847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7e765-f8a2-4cd3-8866-0234b33119e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:38.478781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7e765-f8a2-4cd3-8866-0234b33119e4 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:38.500044 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1048/2094] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:33:38 2022] POST /identity/v3/users => generated 302 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:33:38.504337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2789932-c445-4cbf-b1e0-b3933f585a62 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:38.516032 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2789932-c445-4cbf-b1e0-b3933f585a62 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:38.521391 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b7182-328b-40c0-9ae4-570d66e72fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:38.522843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b7182-328b-40c0-9ae4-570d66e72fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.523133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b7182-328b-40c0-9ae4-570d66e72fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.523408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b7182-328b-40c0-9ae4-570d66e72fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.524595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b7182-328b-40c0-9ae4-570d66e72fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:38.525739 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b7182-328b-40c0-9ae4-570d66e72fe7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:38.536374 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1047/2095] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:38.541006 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7e765-f8a2-4cd3-8866-0234b33119e4 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:38.551850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7e765-f8a2-4cd3-8866-0234b33119e4 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:38.557188 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4d145-4515-47e2-9628-78acfc473ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:38.557785 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4d145-4515-47e2-9628-78acfc473ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.558059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4d145-4515-47e2-9628-78acfc473ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.558323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4d145-4515-47e2-9628-78acfc473ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.558959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4d145-4515-47e2-9628-78acfc473ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:38.560553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4d145-4515-47e2-9628-78acfc473ca5 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:38.572272 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1049/2096] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:33:38.576865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b7182-328b-40c0-9ae4-570d66e72fe7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:38.588567 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b7182-328b-40c0-9ae4-570d66e72fe7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:38.593510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad354c-8d77-43b3-8018-40ea58c620d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:38.594168 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad354c-8d77-43b3-8018-40ea58c620d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.594452 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad354c-8d77-43b3-8018-40ea58c620d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.594965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad354c-8d77-43b3-8018-40ea58c620d3 admin admin] PATH_INFO: `/v3/projects/66453dd19d974d4a80f57122dac1573a/users/3b88eeb7ac014ac8a65edc73d548219b/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.609947 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad354c-8d77-43b3-8018-40ea58c620d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66453dd19d974d4a80f57122dac1573a, user_id=3b88eeb7ac014ac8a65edc73d548219b, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:38.612580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad354c-8d77-43b3-8018-40ea58c620d3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:38.627969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1048/2097] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:33:38 2022] PUT /identity/v3/projects/66453dd19d974d4a80f57122dac1573a/users/3b88eeb7ac014ac8a65edc73d548219b/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:38.638483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304b12-b329-4884-8166-4df9bbb3bc58 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.638857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304b12-b329-4884-8166-4df9bbb3bc58 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.639204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304b12-b329-4884-8166-4df9bbb3bc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.677010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-0c304b12-b329-4884-8166-4df9bbb3bc58 None None] MFA Rules not processed for user `3b88eeb7ac014ac8a65edc73d548219b`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:38.679275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0c304b12-b329-4884-8166-4df9bbb3bc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:38.689076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:38.748914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1050/2098] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:38 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:38.753275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad354c-8d77-43b3-8018-40ea58c620d3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:38.816090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad354c-8d77-43b3-8018-40ea58c620d3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:38.821324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c278e-8095-4860-9974-4c458b4461e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:38.821923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c278e-8095-4860-9974-4c458b4461e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.822198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c278e-8095-4860-9974-4c458b4461e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.822466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c278e-8095-4860-9974-4c458b4461e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.823105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c278e-8095-4860-9974-4c458b4461e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:38.825043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c278e-8095-4860-9974-4c458b4461e3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:38.851303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1049/2099] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:33:38 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:33:38.855394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304b12-b329-4884-8166-4df9bbb3bc58 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:38.866573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304b12-b329-4884-8166-4df9bbb3bc58 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:38.922256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f34f5c-bb82-4ded-bc5a-0a3b4971be60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:38.922853 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f34f5c-bb82-4ded-bc5a-0a3b4971be60 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.923130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f34f5c-bb82-4ded-bc5a-0a3b4971be60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.923416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f34f5c-bb82-4ded-bc5a-0a3b4971be60 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.924025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f34f5c-bb82-4ded-bc5a-0a3b4971be60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:38.925950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f34f5c-bb82-4ded-bc5a-0a3b4971be60 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:38.947725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1051/2100] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:33:38 2022] POST /identity/v3/users => generated 308 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:33:38.951667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c278e-8095-4860-9974-4c458b4461e3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:38.962350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c278e-8095-4860-9974-4c458b4461e3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:38.967213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f388f-4c30-440e-a299-e53f329616c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:38.967838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f388f-4c30-440e-a299-e53f329616c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:38.968088 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f388f-4c30-440e-a299-e53f329616c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:38.968383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f388f-4c30-440e-a299-e53f329616c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:38.969084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f388f-4c30-440e-a299-e53f329616c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:38.970682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f388f-4c30-440e-a299-e53f329616c0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:38.981819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1050/2101] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:38.986848 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f34f5c-bb82-4ded-bc5a-0a3b4971be60 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:38.997437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f34f5c-bb82-4ded-bc5a-0a3b4971be60 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:39.002208 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e36f8d-1b5f-48b4-b29e-3515e5523868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:39.002849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e36f8d-1b5f-48b4-b29e-3515e5523868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:39.003129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e36f8d-1b5f-48b4-b29e-3515e5523868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:39.003397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e36f8d-1b5f-48b4-b29e-3515e5523868 admin admin] PATH_INFO: `/v3/projects/7fb60f781df3483797e3858972636900/users/9ab3c86d46fa4f369a9ed7130e83a35c/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:39.018018 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e36f8d-1b5f-48b4-b29e-3515e5523868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb60f781df3483797e3858972636900, user_id=9ab3c86d46fa4f369a9ed7130e83a35c, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:39.020637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e36f8d-1b5f-48b4-b29e-3515e5523868 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:39.033137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1052/2102] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:33:38 2022] PUT /identity/v3/projects/7fb60f781df3483797e3858972636900/users/9ab3c86d46fa4f369a9ed7130e83a35c/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:39.037096 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f388f-4c30-440e-a299-e53f329616c0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:39.047920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f388f-4c30-440e-a299-e53f329616c0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:39.072445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:39.107017 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9d30c-9252-4caf-aa4b-9f920affe8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:39.107616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9d30c-9252-4caf-aa4b-9f920affe8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:39.107892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9d30c-9252-4caf-aa4b-9f920affe8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:39.108162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9d30c-9252-4caf-aa4b-9f920affe8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:39.108814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e9d30c-9252-4caf-aa4b-9f920affe8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:39.110410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e9d30c-9252-4caf-aa4b-9f920affe8f3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:39.120436 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1051/2103] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:39.124565 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e36f8d-1b5f-48b4-b29e-3515e5523868 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:39.134743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e36f8d-1b5f-48b4-b29e-3515e5523868 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:39.139224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83ffaa-ba9c-4ca1-9e42-8c0da0fb20ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:39.139879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83ffaa-ba9c-4ca1-9e42-8c0da0fb20ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:39.140155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83ffaa-ba9c-4ca1-9e42-8c0da0fb20ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:39.140422 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83ffaa-ba9c-4ca1-9e42-8c0da0fb20ab admin admin] PATH_INFO: `/v3/projects/7fb60f781df3483797e3858972636900/users/9ab3c86d46fa4f369a9ed7130e83a35c/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:39.142318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83ffaa-ba9c-4ca1-9e42-8c0da0fb20ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb60f781df3483797e3858972636900, user_id=9ab3c86d46fa4f369a9ed7130e83a35c, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:39.144964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83ffaa-ba9c-4ca1-9e42-8c0da0fb20ab admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:39.155022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1053/2104] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:33:39 2022] PUT /identity/v3/projects/7fb60f781df3483797e3858972636900/users/9ab3c86d46fa4f369a9ed7130e83a35c/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:39.168155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6eeb43-ef54-48e6-a014-5a6fe8d77292 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:39.168424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6eeb43-ef54-48e6-a014-5a6fe8d77292 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:39.168698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6eeb43-ef54-48e6-a014-5a6fe8d77292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:39.196267 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3a6eeb43-ef54-48e6-a014-5a6fe8d77292 None None] MFA Rules not processed for user `9ab3c86d46fa4f369a9ed7130e83a35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:39.197514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3a6eeb43-ef54-48e6-a014-5a6fe8d77292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:39.263506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1052/2105] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:39 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:39.274749 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83ffaa-ba9c-4ca1-9e42-8c0da0fb20ab admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:39.276467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-aa83ffaa-ba9c-4ca1-9e42-8c0da0fb20ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:39.335609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83ffaa-ba9c-4ca1-9e42-8c0da0fb20ab admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:39.339224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdba5db-0162-42a5-ab73-ffe46f00ec83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:39.340073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdba5db-0162-42a5-ab73-ffe46f00ec83 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:39.340073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdba5db-0162-42a5-ab73-ffe46f00ec83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:39.340359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdba5db-0162-42a5-ab73-ffe46f00ec83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:39.349698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdba5db-0162-42a5-ab73-ffe46f00ec83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:39.351904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdba5db-0162-42a5-ab73-ffe46f00ec83 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:39.361897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1054/2106] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:39 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:39.535261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6eeb43-ef54-48e6-a014-5a6fe8d77292 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:39.611836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6eeb43-ef54-48e6-a014-5a6fe8d77292 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:39.618560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:39.619238 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:39.619523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:39.619795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] PATH_INFO: `/v3/users/3b88eeb7ac014ac8a65edc73d548219b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:39.620957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b88eeb7ac014ac8a65edc73d548219b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:39.623331 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:39.689158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b88eeb7ac014ac8a65edc73d548219b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:39.690432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b88eeb7ac014ac8a65edc73d548219b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:39.691308 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b88eeb7ac014ac8a65edc73d548219b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:39.694388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:39.710765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b88eeb7ac014ac8a65edc73d548219b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:39.718877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b88eeb7ac014ac8a65edc73d548219b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:39.749244 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1053/2107] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:33:39 2022] DELETE /identity/v3/users/3b88eeb7ac014ac8a65edc73d548219b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:39.754601 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67126e5c-e765-46c1-8b6a-9e9fef2a3623 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:39.755101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67126e5c-e765-46c1-8b6a-9e9fef2a3623 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:39.755208 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67126e5c-e765-46c1-8b6a-9e9fef2a3623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:39.763456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:39.776212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-67126e5c-e765-46c1-8b6a-9e9fef2a3623 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:39.777463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-67126e5c-e765-46c1-8b6a-9e9fef2a3623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:39.872202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1055/2108] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:39.879305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:39.881909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:39.947113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e26d3e-f607-4b74-9d05-1654724a1e2d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:39.950940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a3158-57b2-4802-b58c-44aa187b9c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:39.951553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a3158-57b2-4802-b58c-44aa187b9c55 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:39.951838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a3158-57b2-4802-b58c-44aa187b9c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:39.952869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a3158-57b2-4802-b58c-44aa187b9c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:39.963299 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a3158-57b2-4802-b58c-44aa187b9c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:39.964880 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a3158-57b2-4802-b58c-44aa187b9c55 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:39.974614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1054/2109] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:40.246991 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67126e5c-e765-46c1-8b6a-9e9fef2a3623 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:40.248414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-67126e5c-e765-46c1-8b6a-9e9fef2a3623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:40.261009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67126e5c-e765-46c1-8b6a-9e9fef2a3623 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:40.268897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:40.269551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:40.269825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:40.270094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] PATH_INFO: `/v3/projects/66453dd19d974d4a80f57122dac1573a` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:40.271521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66453dd19d974d4a80f57122dac1573a)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:40.274004 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:40.325570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66453dd19d974d4a80f57122dac1573a', 'request_id': 'req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:40.331836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66453dd19d974d4a80f57122dac1573a', 'request_id': 'req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:40.338856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66453dd19d974d4a80f57122dac1573a', 'request_id': 'req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:40.353258 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] The token cache is being invalidate because project 66453dd19d974d4a80f57122dac1573a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:33:40.353540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:40.354169 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:40.355685 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1056/2110] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:33:40 2022] DELETE /identity/v3/projects/66453dd19d974d4a80f57122dac1573a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:33:40.359648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a3158-57b2-4802-b58c-44aa187b9c55 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:40.369416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a3158-57b2-4802-b58c-44aa187b9c55 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:40.428473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b81770-0c79-42d6-95ad-e304c04f4cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:40.429192 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b81770-0c79-42d6-95ad-e304c04f4cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:40.429474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b81770-0c79-42d6-95ad-e304c04f4cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:40.429756 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b81770-0c79-42d6-95ad-e304c04f4cdc admin admin] PATH_INFO: `/v3/users/9ab3c86d46fa4f369a9ed7130e83a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:40.430947 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b81770-0c79-42d6-95ad-e304c04f4cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab3c86d46fa4f369a9ed7130e83a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:40.432941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b81770-0c79-42d6-95ad-e304c04f4cdc admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:40.489583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-00b81770-0c79-42d6-95ad-e304c04f4cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab3c86d46fa4f369a9ed7130e83a35c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:40.490340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-00b81770-0c79-42d6-95ad-e304c04f4cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab3c86d46fa4f369a9ed7130e83a35c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:40.491400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-00b81770-0c79-42d6-95ad-e304c04f4cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab3c86d46fa4f369a9ed7130e83a35c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:40.532346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-00b81770-0c79-42d6-95ad-e304c04f4cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab3c86d46fa4f369a9ed7130e83a35c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:40.541408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:40.547147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-00b81770-0c79-42d6-95ad-e304c04f4cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab3c86d46fa4f369a9ed7130e83a35c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:40.580472 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1055/2111] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:33:40 2022] DELETE /identity/v3/users/9ab3c86d46fa4f369a9ed7130e83a35c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:40.896664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:40.898169 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:40.913697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83e0be-1fa2-4518-9ff3-fee49abb7e18 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:40.972928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1ab3c-1854-4807-9615-e72047c3e1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:40.973680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1ab3c-1854-4807-9615-e72047c3e1ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:40.973964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1ab3c-1854-4807-9615-e72047c3e1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:40.974231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1ab3c-1854-4807-9615-e72047c3e1ab admin admin] PATH_INFO: `/v3/projects/7fb60f781df3483797e3858972636900` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:40.975590 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1ab3c-1854-4807-9615-e72047c3e1ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb60f781df3483797e3858972636900)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:40.977540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1ab3c-1854-4807-9615-e72047c3e1ab admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:41.021441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:41.036129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-19f1ab3c-1854-4807-9615-e72047c3e1ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb60f781df3483797e3858972636900', 'request_id': 'req-19f1ab3c-1854-4807-9615-e72047c3e1ab'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:41.042334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-19f1ab3c-1854-4807-9615-e72047c3e1ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb60f781df3483797e3858972636900', 'request_id': 'req-19f1ab3c-1854-4807-9615-e72047c3e1ab'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:41.048526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-19f1ab3c-1854-4807-9615-e72047c3e1ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb60f781df3483797e3858972636900', 'request_id': 'req-19f1ab3c-1854-4807-9615-e72047c3e1ab'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:41.062057 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-19f1ab3c-1854-4807-9615-e72047c3e1ab admin admin] The token cache is being invalidate because project 7fb60f781df3483797e3858972636900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:33:41.062460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-19f1ab3c-1854-4807-9615-e72047c3e1ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:41.063156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-19f1ab3c-1854-4807-9615-e72047c3e1ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:33:41.064705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1057/2112] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:33:40 2022] DELETE /identity/v3/projects/7fb60f781df3483797e3858972636900 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:41.070220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769df1f-8566-4789-a658-8a3ff6da8268 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.070677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769df1f-8566-4789-a658-8a3ff6da8268 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.070759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769df1f-8566-4789-a658-8a3ff6da8268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.090804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5769df1f-8566-4789-a658-8a3ff6da8268 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:41.092039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5769df1f-8566-4789-a658-8a3ff6da8268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:41.164945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1056/2113] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:41.172579 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299f1e7-7b58-4802-b951-de72355fe4aa None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.173581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299f1e7-7b58-4802-b951-de72355fe4aa None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.173965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299f1e7-7b58-4802-b951-de72355fe4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.199523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f299f1e7-7b58-4802-b951-de72355fe4aa None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:41.201175 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f299f1e7-7b58-4802-b951-de72355fe4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:41.215304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1058/2114] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:41.219631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769df1f-8566-4789-a658-8a3ff6da8268 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:41.232218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769df1f-8566-4789-a658-8a3ff6da8268 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:41.239189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a476d-81eb-4732-b3d1-52709323e9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:41.241832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a476d-81eb-4732-b3d1-52709323e9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.242817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a476d-81eb-4732-b3d1-52709323e9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.243430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a476d-81eb-4732-b3d1-52709323e9f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.244853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a476d-81eb-4732-b3d1-52709323e9f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:33:41.245320 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a476d-81eb-4732-b3d1-52709323e9f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:41.248389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a476d-81eb-4732-b3d1-52709323e9f0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:41.269246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1057/2115] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:33:41.270677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299f1e7-7b58-4802-b951-de72355fe4aa None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:41.282868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299f1e7-7b58-4802-b951-de72355fe4aa None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:41.288640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0019ae-6146-4f1b-a92a-e9d5beaa82a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:41.289505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0019ae-6146-4f1b-a92a-e9d5beaa82a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.289881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0019ae-6146-4f1b-a92a-e9d5beaa82a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.290247 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0019ae-6146-4f1b-a92a-e9d5beaa82a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.291022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0019ae-6146-4f1b-a92a-e9d5beaa82a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:41.293269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0019ae-6146-4f1b-a92a-e9d5beaa82a9 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:41.331294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1059/2116] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:33:41 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:33:41.332728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a476d-81eb-4732-b3d1-52709323e9f0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:41.344013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a476d-81eb-4732-b3d1-52709323e9f0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:41.408257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07499e0c-be13-4d6b-898a-5e797c1ab688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:41.408905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07499e0c-be13-4d6b-898a-5e797c1ab688 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.409199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07499e0c-be13-4d6b-898a-5e797c1ab688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.409456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07499e0c-be13-4d6b-898a-5e797c1ab688 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.410074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07499e0c-be13-4d6b-898a-5e797c1ab688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:41.411991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07499e0c-be13-4d6b-898a-5e797c1ab688 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:41.435590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1058/2117] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:33:41 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:33:41.439498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0019ae-6146-4f1b-a92a-e9d5beaa82a9 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:41.450312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0019ae-6146-4f1b-a92a-e9d5beaa82a9 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:41.455365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce952023-aeb9-488b-8857-803071b1400e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:41.456071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce952023-aeb9-488b-8857-803071b1400e admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.456423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce952023-aeb9-488b-8857-803071b1400e admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.456795 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce952023-aeb9-488b-8857-803071b1400e admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.457762 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce952023-aeb9-488b-8857-803071b1400e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:41.459446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce952023-aeb9-488b-8857-803071b1400e admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:41.469750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1060/2118] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:41.473685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07499e0c-be13-4d6b-898a-5e797c1ab688 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:41.484336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07499e0c-be13-4d6b-898a-5e797c1ab688 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:41.489051 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89fc6a-8008-4ba4-a3c1-c02942ab4455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:41.489645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89fc6a-8008-4ba4-a3c1-c02942ab4455 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.489931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89fc6a-8008-4ba4-a3c1-c02942ab4455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.490208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89fc6a-8008-4ba4-a3c1-c02942ab4455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.491614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89fc6a-8008-4ba4-a3c1-c02942ab4455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:41.493406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89fc6a-8008-4ba4-a3c1-c02942ab4455 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:41.503787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1059/2119] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:41.507820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce952023-aeb9-488b-8857-803071b1400e admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:41.517872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce952023-aeb9-488b-8857-803071b1400e admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:41.523206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e6106-d26c-4f55-9992-808562046a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:41.523793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e6106-d26c-4f55-9992-808562046a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.524074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e6106-d26c-4f55-9992-808562046a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.524385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e6106-d26c-4f55-9992-808562046a7c admin admin] PATH_INFO: `/v3/projects/2838453590e1407a9f494119f1f0b6a8/users/d8eb0f648b1a44978c5bbe4caced3498/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.539536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e6106-d26c-4f55-9992-808562046a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2838453590e1407a9f494119f1f0b6a8, user_id=d8eb0f648b1a44978c5bbe4caced3498, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:41.542233 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e6106-d26c-4f55-9992-808562046a7c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:41.555429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1061/2120] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:33:41 2022] PUT /identity/v3/projects/2838453590e1407a9f494119f1f0b6a8/users/d8eb0f648b1a44978c5bbe4caced3498/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:41.565689 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431241f8-8d92-46b7-806d-69e2a0b855b6 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.565992 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431241f8-8d92-46b7-806d-69e2a0b855b6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.566286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431241f8-8d92-46b7-806d-69e2a0b855b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.595809 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-431241f8-8d92-46b7-806d-69e2a0b855b6 None None] MFA Rules not processed for user `d8eb0f648b1a44978c5bbe4caced3498`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:41.597114 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-431241f8-8d92-46b7-806d-69e2a0b855b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:41.642154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:41.665945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1060/2121] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:41.670502 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e6106-d26c-4f55-9992-808562046a7c admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:41.736744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e6106-d26c-4f55-9992-808562046a7c admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:41.741800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d52fed-eef6-45dd-b6d4-ec5431a77b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:41.742451 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d52fed-eef6-45dd-b6d4-ec5431a77b95 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.742740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d52fed-eef6-45dd-b6d4-ec5431a77b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.743011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d52fed-eef6-45dd-b6d4-ec5431a77b95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.743676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d52fed-eef6-45dd-b6d4-ec5431a77b95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:41.745622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d52fed-eef6-45dd-b6d4-ec5431a77b95 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:41.778010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1062/2122] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:33:41 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:33:41.779227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431241f8-8d92-46b7-806d-69e2a0b855b6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:41.790743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431241f8-8d92-46b7-806d-69e2a0b855b6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:41.852651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ce3e7-83a0-40d7-ba02-b200f935da8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:41.853321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ce3e7-83a0-40d7-ba02-b200f935da8d admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.853597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ce3e7-83a0-40d7-ba02-b200f935da8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.853869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ce3e7-83a0-40d7-ba02-b200f935da8d admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.854484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ce3e7-83a0-40d7-ba02-b200f935da8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:41.856406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ce3e7-83a0-40d7-ba02-b200f935da8d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:41.878050 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1061/2123] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:33:41 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:33:41.879793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d52fed-eef6-45dd-b6d4-ec5431a77b95 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:41.890293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d52fed-eef6-45dd-b6d4-ec5431a77b95 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:41.894983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7eeefd-e668-4c84-8ae4-d09ef77b1273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:41.895594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7eeefd-e668-4c84-8ae4-d09ef77b1273 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.895880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7eeefd-e668-4c84-8ae4-d09ef77b1273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.896155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7eeefd-e668-4c84-8ae4-d09ef77b1273 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.897343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7eeefd-e668-4c84-8ae4-d09ef77b1273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:41.899056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7eeefd-e668-4c84-8ae4-d09ef77b1273 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:41.910841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1063/2124] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:41.915612 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ce3e7-83a0-40d7-ba02-b200f935da8d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:41.926677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ce3e7-83a0-40d7-ba02-b200f935da8d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:41.931491 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea1774-ba95-4cd7-805b-8e5849faae08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:41.932153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea1774-ba95-4cd7-805b-8e5849faae08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:41.932443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea1774-ba95-4cd7-805b-8e5849faae08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:41.932716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea1774-ba95-4cd7-805b-8e5849faae08 admin admin] PATH_INFO: `/v3/projects/e1ca258238a2423db8c0b0345106b9c5/users/e6f29b7ae2a44b99bc52c69557ef1ef2/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:41.948589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea1774-ba95-4cd7-805b-8e5849faae08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ca258238a2423db8c0b0345106b9c5, user_id=e6f29b7ae2a44b99bc52c69557ef1ef2, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:41.951270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea1774-ba95-4cd7-805b-8e5849faae08 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:41.966302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1062/2125] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:33:41 2022] PUT /identity/v3/projects/e1ca258238a2423db8c0b0345106b9c5/users/e6f29b7ae2a44b99bc52c69557ef1ef2/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:41.970020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7eeefd-e668-4c84-8ae4-d09ef77b1273 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:41.980162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7eeefd-e668-4c84-8ae4-d09ef77b1273 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:42.006977 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:42.042219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab40cf4-d10c-431b-857b-294d7192a391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:42.042825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40cf4-d10c-431b-857b-294d7192a391 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:42.043102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40cf4-d10c-431b-857b-294d7192a391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:42.043371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40cf4-d10c-431b-857b-294d7192a391 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:42.044006 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab40cf4-d10c-431b-857b-294d7192a391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:42.045623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab40cf4-d10c-431b-857b-294d7192a391 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:42.056789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1064/2126] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:33:42.060022 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea1774-ba95-4cd7-805b-8e5849faae08 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:42.071269 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea1774-ba95-4cd7-805b-8e5849faae08 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:42.075935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d38309-bd9f-40df-8435-5d856cb23a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:42.076593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d38309-bd9f-40df-8435-5d856cb23a37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:42.076908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d38309-bd9f-40df-8435-5d856cb23a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:42.077179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d38309-bd9f-40df-8435-5d856cb23a37 admin admin] PATH_INFO: `/v3/projects/e1ca258238a2423db8c0b0345106b9c5/users/e6f29b7ae2a44b99bc52c69557ef1ef2/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:42.079088 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d38309-bd9f-40df-8435-5d856cb23a37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ca258238a2423db8c0b0345106b9c5, user_id=e6f29b7ae2a44b99bc52c69557ef1ef2, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:42.081783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d38309-bd9f-40df-8435-5d856cb23a37 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:42.093051 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1063/2127] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:33:42 2022] PUT /identity/v3/projects/e1ca258238a2423db8c0b0345106b9c5/users/e6f29b7ae2a44b99bc52c69557ef1ef2/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:33:42.104223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15368e23-8b76-46d3-97b2-a6d553025d9c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:42.104502 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15368e23-8b76-46d3-97b2-a6d553025d9c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:42.104803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15368e23-8b76-46d3-97b2-a6d553025d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:42.133292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-15368e23-8b76-46d3-97b2-a6d553025d9c None None] MFA Rules not processed for user `e6f29b7ae2a44b99bc52c69557ef1ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:33:42.134669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-15368e23-8b76-46d3-97b2-a6d553025d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:42.205393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1065/2128] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:33:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:33:42.217679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d38309-bd9f-40df-8435-5d856cb23a37 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:42.219356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-22d38309-bd9f-40df-8435-5d856cb23a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:33:42.276594 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d38309-bd9f-40df-8435-5d856cb23a37 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:42.280547 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bdc94-7c5d-46ff-a725-933561fad541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:42.281144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bdc94-7c5d-46ff-a725-933561fad541 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:42.281423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bdc94-7c5d-46ff-a725-933561fad541 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:42.281693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bdc94-7c5d-46ff-a725-933561fad541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:42.291201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bdc94-7c5d-46ff-a725-933561fad541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:42.292626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bdc94-7c5d-46ff-a725-933561fad541 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:42.302285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1064/2129] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:42.381816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15368e23-8b76-46d3-97b2-a6d553025d9c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:33:42.391305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15368e23-8b76-46d3-97b2-a6d553025d9c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:33:42.395013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5322a-3683-46b9-a149-510bbad8983c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:33:42.395618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5322a-3683-46b9-a149-510bbad8983c service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:33:42.395900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5322a-3683-46b9-a149-510bbad8983c service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:33:42.396171 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5322a-3683-46b9-a149-510bbad8983c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:33:42.456876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5322a-3683-46b9-a149-510bbad8983c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:33:42.458644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5322a-3683-46b9-a149-510bbad8983c service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:33:42.468336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1066/2130] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:33:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:33:52.297867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:33:52.463809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=78 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:03.865595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea37a16-ece2-4bf7-a70a-9b7ecde37ba7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:03.866250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea37a16-ece2-4bf7-a70a-9b7ecde37ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:03.866250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea37a16-ece2-4bf7-a70a-9b7ecde37ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:03.886316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4ea37a16-ece2-4bf7-a70a-9b7ecde37ba7 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:03.887531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4ea37a16-ece2-4bf7-a70a-9b7ecde37ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:03.961648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1065/2131] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:34:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:04.018860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5322a-3683-46b9-a149-510bbad8983c service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:04.020458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5fc5322a-3683-46b9-a149-510bbad8983c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:04.113139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5322a-3683-46b9-a149-510bbad8983c service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:04.120438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e590b-3015-448a-aac7-71bdd75bd834 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:04.121796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e590b-3015-448a-aac7-71bdd75bd834 service swift] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:04.122083 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e590b-3015-448a-aac7-71bdd75bd834 service swift] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:04.122470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e590b-3015-448a-aac7-71bdd75bd834 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:04.133130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e590b-3015-448a-aac7-71bdd75bd834 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:04.134763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e590b-3015-448a-aac7-71bdd75bd834 service swift] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:04.143003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1067/2132] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:34:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:34:13.952777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=25 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:14.139589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=32 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:16.936259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b23e-9491-4614-a8fa-46aed3bf51d3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:16.936741 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b23e-9491-4614-a8fa-46aed3bf51d3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:16.937190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b23e-9491-4614-a8fa-46aed3bf51d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:16.968629 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6e12b23e-9491-4614-a8fa-46aed3bf51d3 None None] MFA Rules not processed for user `7936bf23a031418b8ce304aa3bbcf8a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:16.970203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6e12b23e-9491-4614-a8fa-46aed3bf51d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:17.030609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1066/2133] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:34:27.023502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=25 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:29.434891 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e590b-3015-448a-aac7-71bdd75bd834 service swift] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:29.535590 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e590b-3015-448a-aac7-71bdd75bd834 service swift] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:29.545627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:29.546521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:29.547111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:29.547724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] PATH_INFO: `/v3/users/d8eb0f648b1a44978c5bbe4caced3498` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:29.549293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8eb0f648b1a44978c5bbe4caced3498)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:29.556184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:29.641916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8eb0f648b1a44978c5bbe4caced3498'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:29.642667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8eb0f648b1a44978c5bbe4caced3498'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:29.643985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8eb0f648b1a44978c5bbe4caced3498'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:29.656081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8eb0f648b1a44978c5bbe4caced3498'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:29.676322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8eb0f648b1a44978c5bbe4caced3498'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:29.708957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1068/2134] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:34:29 2022] DELETE /identity/v3/users/d8eb0f648b1a44978c5bbe4caced3498 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:29.716270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8062645-7fff-4f76-81d1-8a4beafd3436 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:29.720447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8062645-7fff-4f76-81d1-8a4beafd3436 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:29.721576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8062645-7fff-4f76-81d1-8a4beafd3436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:29.796202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c8062645-7fff-4f76-81d1-8a4beafd3436 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:29.798442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c8062645-7fff-4f76-81d1-8a4beafd3436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:29.924023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1067/2135] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:29.938210 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:29.940909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:30.016892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37419ef1-9525-439f-9f8e-1e4ccd026b3c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:30.016892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37419ef1-9525-439f-9f8e-1e4ccd026b3c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:30.017340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37419ef1-9525-439f-9f8e-1e4ccd026b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:30.042560 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aa2b9-6fe5-417c-80ca-e25ca829c422 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:30.048299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5414a7-3f78-48de-ab0a-b3919e03907f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:30.048970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5414a7-3f78-48de-ab0a-b3919e03907f service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:30.049248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5414a7-3f78-48de-ab0a-b3919e03907f service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:30.049518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5414a7-3f78-48de-ab0a-b3919e03907f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:30.051596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-37419ef1-9525-439f-9f8e-1e4ccd026b3c None None] MFA Rules not processed for user `9b0455ae3ec94c929c940df6a0f6687d`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:30.053358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-37419ef1-9525-439f-9f8e-1e4ccd026b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:30.061174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5414a7-3f78-48de-ab0a-b3919e03907f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:30.062814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5414a7-3f78-48de-ab0a-b3919e03907f service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:30.076692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1069/2136] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:34:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:34:30.130755 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1068/2137] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:30.488516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5414a7-3f78-48de-ab0a-b3919e03907f service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:30.490761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8f5414a7-3f78-48de-ab0a-b3919e03907f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:30.515280 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5414a7-3f78-48de-ab0a-b3919e03907f service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:30.529129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:30.529570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:30.529847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:30.530117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] PATH_INFO: `/v3/projects/2838453590e1407a9f494119f1f0b6a8` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:30.531437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] RBAC: Authorizing `identity:delete_project(project_id=2838453590e1407a9f494119f1f0b6a8)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:30.533555 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:30.573428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:30.645511 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2838453590e1407a9f494119f1f0b6a8', 'request_id': 'req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:30.660316 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2838453590e1407a9f494119f1f0b6a8', 'request_id': 'req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:30.673529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2838453590e1407a9f494119f1f0b6a8', 'request_id': 'req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:30.702253 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] The token cache is being invalidate because project 2838453590e1407a9f494119f1f0b6a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:34:30.702997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:30.703854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:30.706444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1070/2138] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:34:30 2022] DELETE /identity/v3/projects/2838453590e1407a9f494119f1f0b6a8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:34:30.710194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37419ef1-9525-439f-9f8e-1e4ccd026b3c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:30.724326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37419ef1-9525-439f-9f8e-1e4ccd026b3c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:30.825078 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99279d-3ea5-4036-ac0b-41c2fc2e4043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:30.825981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99279d-3ea5-4036-ac0b-41c2fc2e4043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:30.826444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99279d-3ea5-4036-ac0b-41c2fc2e4043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:30.826909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99279d-3ea5-4036-ac0b-41c2fc2e4043 admin admin] PATH_INFO: `/v3/users/e6f29b7ae2a44b99bc52c69557ef1ef2` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:30.828324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99279d-3ea5-4036-ac0b-41c2fc2e4043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f29b7ae2a44b99bc52c69557ef1ef2)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:30.831398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99279d-3ea5-4036-ac0b-41c2fc2e4043 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:30.936465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-4e99279d-3ea5-4036-ac0b-41c2fc2e4043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f29b7ae2a44b99bc52c69557ef1ef2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:30.937954 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-4e99279d-3ea5-4036-ac0b-41c2fc2e4043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f29b7ae2a44b99bc52c69557ef1ef2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:30.939323 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-4e99279d-3ea5-4036-ac0b-41c2fc2e4043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f29b7ae2a44b99bc52c69557ef1ef2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:30.940839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:30.966325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-4e99279d-3ea5-4036-ac0b-41c2fc2e4043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f29b7ae2a44b99bc52c69557ef1ef2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:30.980178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-4e99279d-3ea5-4036-ac0b-41c2fc2e4043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f29b7ae2a44b99bc52c69557ef1ef2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:31.012808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1069/2139] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:34:30 2022] DELETE /identity/v3/users/e6f29b7ae2a44b99bc52c69557ef1ef2 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:31.439617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:31.441505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:31.475293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c8b4f-ddd8-4a31-9b3f-8bbd6aed3bed admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:31.559602 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e84634-2d05-434d-bacf-b92b20b914cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:31.561022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e84634-2d05-434d-bacf-b92b20b914cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:31.561299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e84634-2d05-434d-bacf-b92b20b914cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:31.561790 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e84634-2d05-434d-bacf-b92b20b914cd admin admin] PATH_INFO: `/v3/projects/e1ca258238a2423db8c0b0345106b9c5` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:31.562927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e84634-2d05-434d-bacf-b92b20b914cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1ca258238a2423db8c0b0345106b9c5)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:31.565017 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e84634-2d05-434d-bacf-b92b20b914cd admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:31.651999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-50e84634-2d05-434d-bacf-b92b20b914cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1ca258238a2423db8c0b0345106b9c5', 'request_id': 'req-50e84634-2d05-434d-bacf-b92b20b914cd'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:31.658971 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-50e84634-2d05-434d-bacf-b92b20b914cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1ca258238a2423db8c0b0345106b9c5', 'request_id': 'req-50e84634-2d05-434d-bacf-b92b20b914cd'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:31.665849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-50e84634-2d05-434d-bacf-b92b20b914cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1ca258238a2423db8c0b0345106b9c5', 'request_id': 'req-50e84634-2d05-434d-bacf-b92b20b914cd'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:31.691577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-50e84634-2d05-434d-bacf-b92b20b914cd admin admin] The token cache is being invalidate because project e1ca258238a2423db8c0b0345106b9c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:34:31.691877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-50e84634-2d05-434d-bacf-b92b20b914cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:31.692537 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-50e84634-2d05-434d-bacf-b92b20b914cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:31.694543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1071/2140] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:34:31 2022] DELETE /identity/v3/projects/e1ca258238a2423db8c0b0345106b9c5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:34:31.700650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a02c8-fdab-4aec-8834-b3a09ee57642 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:31.701106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a02c8-fdab-4aec-8834-b3a09ee57642 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:31.701518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a02c8-fdab-4aec-8834-b3a09ee57642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:31.723796 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-585a02c8-fdab-4aec-8834-b3a09ee57642 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:31.726537 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-585a02c8-fdab-4aec-8834-b3a09ee57642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:31.796228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1070/2141] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:31.807712 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1fe9c-16e2-449d-ae9d-bc04b838fcef None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:31.807987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1fe9c-16e2-449d-ae9d-bc04b838fcef None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:31.808491 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1fe9c-16e2-449d-ae9d-bc04b838fcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:31.843583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-edd1fe9c-16e2-449d-ae9d-bc04b838fcef None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:31.844953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-edd1fe9c-16e2-449d-ae9d-bc04b838fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:31.859883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1072/2142] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:31.863767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a02c8-fdab-4aec-8834-b3a09ee57642 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:31.874611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a02c8-fdab-4aec-8834-b3a09ee57642 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:31.879398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a70b0c-5cae-4c04-b8b3-acc284a42b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:31.880035 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a70b0c-5cae-4c04-b8b3-acc284a42b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:31.880342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a70b0c-5cae-4c04-b8b3-acc284a42b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:31.880613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a70b0c-5cae-4c04-b8b3-acc284a42b57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:31.881322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a70b0c-5cae-4c04-b8b3-acc284a42b57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:34:31.881672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a70b0c-5cae-4c04-b8b3-acc284a42b57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:31.884297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a70b0c-5cae-4c04-b8b3-acc284a42b57 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:31.897711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1071/2143] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:34:31.902285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1fe9c-16e2-449d-ae9d-bc04b838fcef None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:31.915352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1fe9c-16e2-449d-ae9d-bc04b838fcef None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:31.920747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7a6c-6991-40cc-a9e8-783d9f2e427a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:31.921481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7a6c-6991-40cc-a9e8-783d9f2e427a admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:31.921825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7a6c-6991-40cc-a9e8-783d9f2e427a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:31.922305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7a6c-6991-40cc-a9e8-783d9f2e427a admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:31.923099 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad7a6c-6991-40cc-a9e8-783d9f2e427a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:31.925356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad7a6c-6991-40cc-a9e8-783d9f2e427a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:31.955985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1073/2144] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:34:31 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:34:31.960208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a70b0c-5cae-4c04-b8b3-acc284a42b57 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:31.972812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a70b0c-5cae-4c04-b8b3-acc284a42b57 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:32.043498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484871df-6754-4d3f-ae54-8ca77fc5cdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:32.044106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484871df-6754-4d3f-ae54-8ca77fc5cdae admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:32.044391 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484871df-6754-4d3f-ae54-8ca77fc5cdae admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:32.044666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484871df-6754-4d3f-ae54-8ca77fc5cdae admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:32.045625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484871df-6754-4d3f-ae54-8ca77fc5cdae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:32.047676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484871df-6754-4d3f-ae54-8ca77fc5cdae admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:32.093926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1072/2145] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:34:31 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:32.097733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7a6c-6991-40cc-a9e8-783d9f2e427a admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:32.109174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7a6c-6991-40cc-a9e8-783d9f2e427a admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:32.114239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5149b6-1fd2-4df0-9fb0-b47ffa39b12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:32.115003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5149b6-1fd2-4df0-9fb0-b47ffa39b12d admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:32.115474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5149b6-1fd2-4df0-9fb0-b47ffa39b12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:32.115835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5149b6-1fd2-4df0-9fb0-b47ffa39b12d admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:32.117115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5149b6-1fd2-4df0-9fb0-b47ffa39b12d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:32.118720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5149b6-1fd2-4df0-9fb0-b47ffa39b12d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:32.121439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:32.134282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1074/2146] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:32.137636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484871df-6754-4d3f-ae54-8ca77fc5cdae admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:32.150773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484871df-6754-4d3f-ae54-8ca77fc5cdae admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:32.155517 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a004b-fdfb-405c-9b67-ed4a24305d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:32.156342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a004b-fdfb-405c-9b67-ed4a24305d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:32.156777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a004b-fdfb-405c-9b67-ed4a24305d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:32.157083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a004b-fdfb-405c-9b67-ed4a24305d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:32.157740 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a004b-fdfb-405c-9b67-ed4a24305d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:32.159361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a004b-fdfb-405c-9b67-ed4a24305d73 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:32.172999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1073/2147] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:34:32.178153 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5149b6-1fd2-4df0-9fb0-b47ffa39b12d admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:32.200024 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5149b6-1fd2-4df0-9fb0-b47ffa39b12d admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:32.229936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efe7a8-f346-4a1c-b884-f939787452bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:32.230797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efe7a8-f346-4a1c-b884-f939787452bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:32.231083 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efe7a8-f346-4a1c-b884-f939787452bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:32.231360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efe7a8-f346-4a1c-b884-f939787452bd admin admin] PATH_INFO: `/v3/projects/183a9e33fa25425aa560805b0dd84b9a/users/4610f8a9001d4badbdfe7ed0fe60581e/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:32.283820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efe7a8-f346-4a1c-b884-f939787452bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=183a9e33fa25425aa560805b0dd84b9a, user_id=4610f8a9001d4badbdfe7ed0fe60581e, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:32.286769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efe7a8-f346-4a1c-b884-f939787452bd admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:32.316801 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1075/2148] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:32 2022] PUT /identity/v3/projects/183a9e33fa25425aa560805b0dd84b9a/users/4610f8a9001d4badbdfe7ed0fe60581e/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:32.332069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332add4-d395-4f32-aca5-47c33077adf5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:32.332771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332add4-d395-4f32-aca5-47c33077adf5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:32.333267 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332add4-d395-4f32-aca5-47c33077adf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:32.390745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6332add4-d395-4f32-aca5-47c33077adf5 None None] MFA Rules not processed for user `4610f8a9001d4badbdfe7ed0fe60581e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:32.406701 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6332add4-d395-4f32-aca5-47c33077adf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:32.436783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:32.510577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1074/2149] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:34:32.515173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efe7a8-f346-4a1c-b884-f939787452bd admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:32.597900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efe7a8-f346-4a1c-b884-f939787452bd admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:32.603171 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61153312-c830-4cac-8406-e4c1a1b5e9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:32.603784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61153312-c830-4cac-8406-e4c1a1b5e9ac admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:32.604062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61153312-c830-4cac-8406-e4c1a1b5e9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:32.604329 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61153312-c830-4cac-8406-e4c1a1b5e9ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:32.604997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61153312-c830-4cac-8406-e4c1a1b5e9ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:32.607367 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61153312-c830-4cac-8406-e4c1a1b5e9ac admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:32.653146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1076/2150] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:34:32 2022] POST /identity/v3/projects => generated 367 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:32.654648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332add4-d395-4f32-aca5-47c33077adf5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:32.680137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332add4-d395-4f32-aca5-47c33077adf5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:32.780859 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47921bf4-a390-4d75-b54f-9ace19837aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:32.781563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47921bf4-a390-4d75-b54f-9ace19837aed admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:32.781908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47921bf4-a390-4d75-b54f-9ace19837aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:32.782245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47921bf4-a390-4d75-b54f-9ace19837aed admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:32.782997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47921bf4-a390-4d75-b54f-9ace19837aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:32.785370 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47921bf4-a390-4d75-b54f-9ace19837aed admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:32.827214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1075/2151] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:34:32 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:32.828417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61153312-c830-4cac-8406-e4c1a1b5e9ac admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:32.845282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61153312-c830-4cac-8406-e4c1a1b5e9ac admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:32.852050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba18d2-a4b5-455c-9ab0-da2398588453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:32.852739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba18d2-a4b5-455c-9ab0-da2398588453 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:32.853036 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba18d2-a4b5-455c-9ab0-da2398588453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:32.853308 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba18d2-a4b5-455c-9ab0-da2398588453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:32.854482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba18d2-a4b5-455c-9ab0-da2398588453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:32.857020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba18d2-a4b5-455c-9ab0-da2398588453 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:32.876654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1077/2152] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:32.879288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47921bf4-a390-4d75-b54f-9ace19837aed admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:32.892279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47921bf4-a390-4d75-b54f-9ace19837aed admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:32.899941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec549a-8bcb-4372-84f0-bfff2bd8b10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:32.900627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec549a-8bcb-4372-84f0-bfff2bd8b10c admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:32.900997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec549a-8bcb-4372-84f0-bfff2bd8b10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:32.901279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec549a-8bcb-4372-84f0-bfff2bd8b10c admin admin] PATH_INFO: `/v3/projects/e50c81b6680a45408412eb4adc5cf844/users/3db506df215f401980a093d71bf30667/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:32.919595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec549a-8bcb-4372-84f0-bfff2bd8b10c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50c81b6680a45408412eb4adc5cf844, user_id=3db506df215f401980a093d71bf30667, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:32.923282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec549a-8bcb-4372-84f0-bfff2bd8b10c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:32.943673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1076/2153] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:32 2022] PUT /identity/v3/projects/e50c81b6680a45408412eb4adc5cf844/users/3db506df215f401980a093d71bf30667/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:32.948054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba18d2-a4b5-455c-9ab0-da2398588453 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:32.963499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba18d2-a4b5-455c-9ab0-da2398588453 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:33.116854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407855a-e55d-4876-ba8f-903540eeffca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:33.117520 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407855a-e55d-4876-ba8f-903540eeffca admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:33.117892 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407855a-e55d-4876-ba8f-903540eeffca admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:33.118225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407855a-e55d-4876-ba8f-903540eeffca admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:33.118880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407855a-e55d-4876-ba8f-903540eeffca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:33.120665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407855a-e55d-4876-ba8f-903540eeffca admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:33.136414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1078/2154] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:33.137730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec549a-8bcb-4372-84f0-bfff2bd8b10c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:33.151097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec549a-8bcb-4372-84f0-bfff2bd8b10c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:33.157001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae504ec-1eaa-4979-97d1-5348335de4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:33.157796 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae504ec-1eaa-4979-97d1-5348335de4c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:33.158174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae504ec-1eaa-4979-97d1-5348335de4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:33.158995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae504ec-1eaa-4979-97d1-5348335de4c3 admin admin] PATH_INFO: `/v3/projects/e50c81b6680a45408412eb4adc5cf844/users/3db506df215f401980a093d71bf30667/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:33.161957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae504ec-1eaa-4979-97d1-5348335de4c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50c81b6680a45408412eb4adc5cf844, user_id=3db506df215f401980a093d71bf30667, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:33.166002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae504ec-1eaa-4979-97d1-5348335de4c3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:33.190392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1077/2155] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:33 2022] PUT /identity/v3/projects/e50c81b6680a45408412eb4adc5cf844/users/3db506df215f401980a093d71bf30667/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:34:33.200779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c821ce-8ce9-4db4-9e1c-b342ff432efd None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:33.201439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c821ce-8ce9-4db4-9e1c-b342ff432efd None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:33.201891 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c821ce-8ce9-4db4-9e1c-b342ff432efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:33.244741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-03c821ce-8ce9-4db4-9e1c-b342ff432efd None None] MFA Rules not processed for user `3db506df215f401980a093d71bf30667`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:33.246063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-03c821ce-8ce9-4db4-9e1c-b342ff432efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:33.338845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:33.473256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1079/2156] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:33.481985 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae504ec-1eaa-4979-97d1-5348335de4c3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:33.483868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7ae504ec-1eaa-4979-97d1-5348335de4c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:33.643623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae504ec-1eaa-4979-97d1-5348335de4c3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:33.658777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504ea30-5fec-4591-8ccf-ce0bf8c91f59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:33.659661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504ea30-5fec-4591-8ccf-ce0bf8c91f59 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:33.659661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504ea30-5fec-4591-8ccf-ce0bf8c91f59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:33.659927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504ea30-5fec-4591-8ccf-ce0bf8c91f59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:33.692975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504ea30-5fec-4591-8ccf-ce0bf8c91f59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:33.694494 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504ea30-5fec-4591-8ccf-ce0bf8c91f59 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:33.720701 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1078/2157] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:34:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:34:34.272321 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c821ce-8ce9-4db4-9e1c-b342ff432efd None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:34.391157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c821ce-8ce9-4db4-9e1c-b342ff432efd None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:34.396400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:34.397104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:34.397410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:34.398351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] PATH_INFO: `/v3/users/4610f8a9001d4badbdfe7ed0fe60581e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:34.399523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4610f8a9001d4badbdfe7ed0fe60581e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:34.401489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:34.482296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4610f8a9001d4badbdfe7ed0fe60581e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:34.482988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4610f8a9001d4badbdfe7ed0fe60581e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:34.483612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4610f8a9001d4badbdfe7ed0fe60581e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:34.493756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4610f8a9001d4badbdfe7ed0fe60581e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:34.509679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4610f8a9001d4badbdfe7ed0fe60581e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:34.542832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1080/2158] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:34:34 2022] DELETE /identity/v3/users/4610f8a9001d4badbdfe7ed0fe60581e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:34.547268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa91399-ac00-49d7-b146-ee5f5025f68b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:34.547726 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa91399-ac00-49d7-b146-ee5f5025f68b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:34.547818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa91399-ac00-49d7-b146-ee5f5025f68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:34.568897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-baa91399-ac00-49d7-b146-ee5f5025f68b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:34.570808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-baa91399-ac00-49d7-b146-ee5f5025f68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:34.581050 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:34.653340 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1079/2159] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:34.663539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:34.665215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:34.743046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:34.750588 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c6118-b41b-454b-bd36-5d92d0adb077 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:34.757793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba067e-1277-423f-aa48-1c2a191de8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:34.759680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba067e-1277-423f-aa48-1c2a191de8af service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:34.760340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba067e-1277-423f-aa48-1c2a191de8af service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:34.762799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba067e-1277-423f-aa48-1c2a191de8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:34.790720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba067e-1277-423f-aa48-1c2a191de8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:34.792396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba067e-1277-423f-aa48-1c2a191de8af service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:34.817484 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1081/2160] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:34:35.477446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa91399-ac00-49d7-b146-ee5f5025f68b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:35.479679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-baa91399-ac00-49d7-b146-ee5f5025f68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:35.501272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa91399-ac00-49d7-b146-ee5f5025f68b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:35.507318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:35.507975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:35.508259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:35.508535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] PATH_INFO: `/v3/projects/183a9e33fa25425aa560805b0dd84b9a` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:35.510414 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=183a9e33fa25425aa560805b0dd84b9a)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:35.512386 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:35.598591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '183a9e33fa25425aa560805b0dd84b9a', 'request_id': 'req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:35.615265 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '183a9e33fa25425aa560805b0dd84b9a', 'request_id': 'req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:35.630181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '183a9e33fa25425aa560805b0dd84b9a', 'request_id': 'req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:35.665377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] The token cache is being invalidate because project 183a9e33fa25425aa560805b0dd84b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:34:35.666935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:35.667885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:35.670862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1080/2161] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:34:35 2022] DELETE /identity/v3/projects/183a9e33fa25425aa560805b0dd84b9a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:35.675074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba067e-1277-423f-aa48-1c2a191de8af service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:35.686557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba067e-1277-423f-aa48-1c2a191de8af service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:35.775771 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde6f96-7bad-45d0-87d3-d90391bdb64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:35.776142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde6f96-7bad-45d0-87d3-d90391bdb64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:35.776427 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde6f96-7bad-45d0-87d3-d90391bdb64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:35.776814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde6f96-7bad-45d0-87d3-d90391bdb64e admin admin] PATH_INFO: `/v3/users/3db506df215f401980a093d71bf30667` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:35.777899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde6f96-7bad-45d0-87d3-d90391bdb64e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3db506df215f401980a093d71bf30667)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:35.779898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde6f96-7bad-45d0-87d3-d90391bdb64e admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:35.878856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6fde6f96-7bad-45d0-87d3-d90391bdb64e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3db506df215f401980a093d71bf30667'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:35.880234 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6fde6f96-7bad-45d0-87d3-d90391bdb64e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3db506df215f401980a093d71bf30667'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:35.881367 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6fde6f96-7bad-45d0-87d3-d90391bdb64e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3db506df215f401980a093d71bf30667'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:35.899121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6fde6f96-7bad-45d0-87d3-d90391bdb64e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3db506df215f401980a093d71bf30667'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:35.919793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6fde6f96-7bad-45d0-87d3-d90391bdb64e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3db506df215f401980a093d71bf30667'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:35.956838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1082/2162] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:34:35 2022] DELETE /identity/v3/users/3db506df215f401980a093d71bf30667 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:36.704387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:36.706947 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:36.748685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00631f2-28b3-4f1b-8362-3fc5e97f4c5a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:36.826277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafff830-3771-4bf0-854a-a1464b9ff837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:36.827160 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafff830-3771-4bf0-854a-a1464b9ff837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:36.828059 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafff830-3771-4bf0-854a-a1464b9ff837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:36.828946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafff830-3771-4bf0-854a-a1464b9ff837 admin admin] PATH_INFO: `/v3/projects/e50c81b6680a45408412eb4adc5cf844` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:36.830673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafff830-3771-4bf0-854a-a1464b9ff837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e50c81b6680a45408412eb4adc5cf844)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:36.832886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafff830-3771-4bf0-854a-a1464b9ff837 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:36.953183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:36.969157 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-fafff830-3771-4bf0-854a-a1464b9ff837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e50c81b6680a45408412eb4adc5cf844', 'request_id': 'req-fafff830-3771-4bf0-854a-a1464b9ff837'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:36.986033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-fafff830-3771-4bf0-854a-a1464b9ff837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e50c81b6680a45408412eb4adc5cf844', 'request_id': 'req-fafff830-3771-4bf0-854a-a1464b9ff837'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:37.050043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-fafff830-3771-4bf0-854a-a1464b9ff837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e50c81b6680a45408412eb4adc5cf844', 'request_id': 'req-fafff830-3771-4bf0-854a-a1464b9ff837'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:37.106670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-fafff830-3771-4bf0-854a-a1464b9ff837 admin admin] The token cache is being invalidate because project e50c81b6680a45408412eb4adc5cf844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:34:37.107064 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-fafff830-3771-4bf0-854a-a1464b9ff837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:37.111358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-fafff830-3771-4bf0-854a-a1464b9ff837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:37.114673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1081/2163] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:34:36 2022] DELETE /identity/v3/projects/e50c81b6680a45408412eb4adc5cf844 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:37.119981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5239cdc-97d5-44f2-882e-9dd628722093 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:37.120433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5239cdc-97d5-44f2-882e-9dd628722093 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:37.120577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5239cdc-97d5-44f2-882e-9dd628722093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:37.149040 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-a5239cdc-97d5-44f2-882e-9dd628722093 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:37.151266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a5239cdc-97d5-44f2-882e-9dd628722093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:37.248736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1083/2164] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:37.251151 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05dfcd-8079-4087-815e-b9e35aaccba9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:37.251496 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05dfcd-8079-4087-815e-b9e35aaccba9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:37.252266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05dfcd-8079-4087-815e-b9e35aaccba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:37.298444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ad05dfcd-8079-4087-815e-b9e35aaccba9 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:37.302024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ad05dfcd-8079-4087-815e-b9e35aaccba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:37.312785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1082/2165] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:37.317419 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5239cdc-97d5-44f2-882e-9dd628722093 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:37.329434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5239cdc-97d5-44f2-882e-9dd628722093 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:37.336317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0f950-4ead-4853-8dbd-6426816439ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:37.340292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0f950-4ead-4853-8dbd-6426816439ee admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:37.341002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0f950-4ead-4853-8dbd-6426816439ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:37.341434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0f950-4ead-4853-8dbd-6426816439ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:37.342286 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0f950-4ead-4853-8dbd-6426816439ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:34:37.342747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0f950-4ead-4853-8dbd-6426816439ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:37.344600 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0f950-4ead-4853-8dbd-6426816439ee admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:37.348087 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:37.369864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1084/2166] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:34:37.373970 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05dfcd-8079-4087-815e-b9e35aaccba9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:37.391699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05dfcd-8079-4087-815e-b9e35aaccba9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:37.399156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48f693-d940-4d6b-ae3c-a597ea1bb9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:37.400060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48f693-d940-4d6b-ae3c-a597ea1bb9a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:37.400730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48f693-d940-4d6b-ae3c-a597ea1bb9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:37.401281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48f693-d940-4d6b-ae3c-a597ea1bb9a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:37.402431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48f693-d940-4d6b-ae3c-a597ea1bb9a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:37.405940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48f693-d940-4d6b-ae3c-a597ea1bb9a1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:37.481122 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1083/2167] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:34:37 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:37.486109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0f950-4ead-4853-8dbd-6426816439ee admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:37.527952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0f950-4ead-4853-8dbd-6426816439ee admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:37.599869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e1742-bf9e-4aa7-8965-7fbc4362fab4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:37.600833 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e1742-bf9e-4aa7-8965-7fbc4362fab4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:37.601533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e1742-bf9e-4aa7-8965-7fbc4362fab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:37.664137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-079e1742-bf9e-4aa7-8965-7fbc4362fab4 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:37.669230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-079e1742-bf9e-4aa7-8965-7fbc4362fab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:37.715843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb9735-9ac8-4a63-811b-b4fb76b171d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:37.716476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb9735-9ac8-4a63-811b-b4fb76b171d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:37.719078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb9735-9ac8-4a63-811b-b4fb76b171d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:37.719702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb9735-9ac8-4a63-811b-b4fb76b171d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:37.720260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb9735-9ac8-4a63-811b-b4fb76b171d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:37.725284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb9735-9ac8-4a63-811b-b4fb76b171d9 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:37.785534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1085/2168] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:34:37 2022] POST /identity/v3/users => generated 294 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:37.789599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb9735-9ac8-4a63-811b-b4fb76b171d9 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:37.811253 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1084/2169] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:34:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:34:37.818532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb9735-9ac8-4a63-811b-b4fb76b171d9 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:37.829468 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9fdee-7666-47ba-83ab-8a3417e58d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:37.830446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9fdee-7666-47ba-83ab-8a3417e58d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:37.831585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9fdee-7666-47ba-83ab-8a3417e58d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:37.833108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9fdee-7666-47ba-83ab-8a3417e58d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:37.833967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9fdee-7666-47ba-83ab-8a3417e58d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:37.834366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e1742-bf9e-4aa7-8965-7fbc4362fab4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:37.838184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-079e1742-bf9e-4aa7-8965-7fbc4362fab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:37.840209 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9fdee-7666-47ba-83ab-8a3417e58d41 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:37.870359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1086/2170] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:37.872197 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9fdee-7666-47ba-83ab-8a3417e58d41 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:37.901456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9fdee-7666-47ba-83ab-8a3417e58d41 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:37.910833 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e5456-c4cc-41ed-b2c8-6d7ea2d7070c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:37.911607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e5456-c4cc-41ed-b2c8-6d7ea2d7070c admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:37.911984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e5456-c4cc-41ed-b2c8-6d7ea2d7070c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:37.912420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e5456-c4cc-41ed-b2c8-6d7ea2d7070c admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:37.913855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e5456-c4cc-41ed-b2c8-6d7ea2d7070c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:37.916160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e5456-c4cc-41ed-b2c8-6d7ea2d7070c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:37.933640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1087/2171] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:37.937862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e5456-c4cc-41ed-b2c8-6d7ea2d7070c admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:37.950725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e5456-c4cc-41ed-b2c8-6d7ea2d7070c admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:37.957035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe65c6-3299-4795-88d1-c0b70f7e2ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:37.958584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe65c6-3299-4795-88d1-c0b70f7e2ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:37.959263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe65c6-3299-4795-88d1-c0b70f7e2ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:37.959840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe65c6-3299-4795-88d1-c0b70f7e2ffc admin admin] PATH_INFO: `/v3/projects/8e7489c151da46479f9be2218e0a2f5b/users/d747a37351104754a8b2b475273732b4/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:37.981536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe65c6-3299-4795-88d1-c0b70f7e2ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e7489c151da46479f9be2218e0a2f5b, user_id=d747a37351104754a8b2b475273732b4, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:37.986939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe65c6-3299-4795-88d1-c0b70f7e2ffc admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:38.015675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1088/2172] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:37 2022] PUT /identity/v3/projects/8e7489c151da46479f9be2218e0a2f5b/users/d747a37351104754a8b2b475273732b4/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:34:38.028639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b31aa-7726-4896-b384-dd10549bac57 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:38.030353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b31aa-7726-4896-b384-dd10549bac57 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:38.030831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b31aa-7726-4896-b384-dd10549bac57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:38.037711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e1742-bf9e-4aa7-8965-7fbc4362fab4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:38.086643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b12b31aa-7726-4896-b384-dd10549bac57 None None] MFA Rules not processed for user `d747a37351104754a8b2b475273732b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:38.089040 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b12b31aa-7726-4896-b384-dd10549bac57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:38.149002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:38.205562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413f26d-5812-43dd-9f0a-81c239ce6ce7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:38.207961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413f26d-5812-43dd-9f0a-81c239ce6ce7 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:38.209110 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413f26d-5812-43dd-9f0a-81c239ce6ce7 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:38.211641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413f26d-5812-43dd-9f0a-81c239ce6ce7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:38.214763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1089/2173] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:34:38.220384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b31aa-7726-4896-b384-dd10549bac57 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:38.271244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:38.311544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413f26d-5812-43dd-9f0a-81c239ce6ce7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:38.313535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413f26d-5812-43dd-9f0a-81c239ce6ce7 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:38.325388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1085/2174] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:34:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 19 16:34:38.342447 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b31aa-7726-4896-b384-dd10549bac57 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:38.350255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e8de9-58d6-4410-8390-c43119e41573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:38.356151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e8de9-58d6-4410-8390-c43119e41573 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:38.356568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e8de9-58d6-4410-8390-c43119e41573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:38.359084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e8de9-58d6-4410-8390-c43119e41573 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:38.359939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e8de9-58d6-4410-8390-c43119e41573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:38.365756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e8de9-58d6-4410-8390-c43119e41573 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:38.460225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1090/2175] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:34:38 2022] POST /identity/v3/projects => generated 387 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:38.466366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413f26d-5812-43dd-9f0a-81c239ce6ce7 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:38.479726 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413f26d-5812-43dd-9f0a-81c239ce6ce7 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:38.640489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1dd5d-9453-42cc-868b-32d103211e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:38.641143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1dd5d-9453-42cc-868b-32d103211e51 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:38.641425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1dd5d-9453-42cc-868b-32d103211e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:38.641697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1dd5d-9453-42cc-868b-32d103211e51 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:38.642382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1dd5d-9453-42cc-868b-32d103211e51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:38.644422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1dd5d-9453-42cc-868b-32d103211e51 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:38.712312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1086/2176] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:34:38 2022] POST /identity/v3/users => generated 300 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:34:38.712668 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e8de9-58d6-4410-8390-c43119e41573 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:38.735150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e8de9-58d6-4410-8390-c43119e41573 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:38.754249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef802ff3-2f5a-4fc7-b1ec-d35dacc3baf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:38.755020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef802ff3-2f5a-4fc7-b1ec-d35dacc3baf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:38.755284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef802ff3-2f5a-4fc7-b1ec-d35dacc3baf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:38.755564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef802ff3-2f5a-4fc7-b1ec-d35dacc3baf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:38.757008 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef802ff3-2f5a-4fc7-b1ec-d35dacc3baf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:38.765865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef802ff3-2f5a-4fc7-b1ec-d35dacc3baf2 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:38.790953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1091/2177] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:38.794135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1dd5d-9453-42cc-868b-32d103211e51 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:38.824892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1dd5d-9453-42cc-868b-32d103211e51 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:38.848866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdc28a-8fbe-44f5-8cf9-11eed3022091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:38.848866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdc28a-8fbe-44f5-8cf9-11eed3022091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:38.849111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdc28a-8fbe-44f5-8cf9-11eed3022091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:38.850310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdc28a-8fbe-44f5-8cf9-11eed3022091 admin admin] PATH_INFO: `/v3/projects/4f09b1e69b9a4ed58ff78ae6d1bbc87b/users/983b31dcab93430c904b0bbccaedc6b8/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:38.889578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecdc28a-8fbe-44f5-8cf9-11eed3022091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f09b1e69b9a4ed58ff78ae6d1bbc87b, user_id=983b31dcab93430c904b0bbccaedc6b8, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:38.893321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecdc28a-8fbe-44f5-8cf9-11eed3022091 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:38.922711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1087/2178] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:38 2022] PUT /identity/v3/projects/4f09b1e69b9a4ed58ff78ae6d1bbc87b/users/983b31dcab93430c904b0bbccaedc6b8/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:38.926114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef802ff3-2f5a-4fc7-b1ec-d35dacc3baf2 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:38.943921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef802ff3-2f5a-4fc7-b1ec-d35dacc3baf2 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:39.074864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f0e57-f866-4b1a-a094-4b9da1e2cf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:39.076049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f0e57-f866-4b1a-a094-4b9da1e2cf1f admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:39.076510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f0e57-f866-4b1a-a094-4b9da1e2cf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:39.076963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f0e57-f866-4b1a-a094-4b9da1e2cf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:39.077630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f0e57-f866-4b1a-a094-4b9da1e2cf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:39.079259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f0e57-f866-4b1a-a094-4b9da1e2cf1f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:39.091406 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1092/2179] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:39.096057 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdc28a-8fbe-44f5-8cf9-11eed3022091 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:39.108738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdc28a-8fbe-44f5-8cf9-11eed3022091 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:39.122777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc804141-85a3-4ec4-9689-cb7ed9a67b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:39.123468 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc804141-85a3-4ec4-9689-cb7ed9a67b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:39.123764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc804141-85a3-4ec4-9689-cb7ed9a67b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:39.124039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc804141-85a3-4ec4-9689-cb7ed9a67b43 admin admin] PATH_INFO: `/v3/projects/4f09b1e69b9a4ed58ff78ae6d1bbc87b/users/983b31dcab93430c904b0bbccaedc6b8/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:39.127887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc804141-85a3-4ec4-9689-cb7ed9a67b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f09b1e69b9a4ed58ff78ae6d1bbc87b, user_id=983b31dcab93430c904b0bbccaedc6b8, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:39.130586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc804141-85a3-4ec4-9689-cb7ed9a67b43 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:39.172728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1088/2180] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:39 2022] PUT /identity/v3/projects/4f09b1e69b9a4ed58ff78ae6d1bbc87b/users/983b31dcab93430c904b0bbccaedc6b8/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:39.174303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff464b-8c36-4a8b-89dc-3ec372830b0b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:39.174892 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff464b-8c36-4a8b-89dc-3ec372830b0b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:39.174998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff464b-8c36-4a8b-89dc-3ec372830b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:39.208817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3fff464b-8c36-4a8b-89dc-3ec372830b0b None None] MFA Rules not processed for user `983b31dcab93430c904b0bbccaedc6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:39.210021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3fff464b-8c36-4a8b-89dc-3ec372830b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:39.298326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1093/2181] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:39.311462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc804141-85a3-4ec4-9689-cb7ed9a67b43 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:39.313408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cc804141-85a3-4ec4-9689-cb7ed9a67b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:39.375246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:39.389196 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc804141-85a3-4ec4-9689-cb7ed9a67b43 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:39.393007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf716d9-f1ee-4495-b3f1-e3d45d2077bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:39.393624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf716d9-f1ee-4495-b3f1-e3d45d2077bf service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:39.393938 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf716d9-f1ee-4495-b3f1-e3d45d2077bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:39.394180 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf716d9-f1ee-4495-b3f1-e3d45d2077bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:39.406911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf716d9-f1ee-4495-b3f1-e3d45d2077bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:39.408425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf716d9-f1ee-4495-b3f1-e3d45d2077bf service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:39.421278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1089/2182] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:34:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:34:39.502796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff464b-8c36-4a8b-89dc-3ec372830b0b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:39.533140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:39.577989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff464b-8c36-4a8b-89dc-3ec372830b0b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:39.583199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:39.584053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:39.584275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:39.584683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] PATH_INFO: `/v3/users/d747a37351104754a8b2b475273732b4` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:39.586501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d747a37351104754a8b2b475273732b4)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:39.589161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:39.661610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd747a37351104754a8b2b475273732b4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:39.662300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd747a37351104754a8b2b475273732b4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:39.662917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd747a37351104754a8b2b475273732b4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:39.670318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd747a37351104754a8b2b475273732b4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:39.684170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd747a37351104754a8b2b475273732b4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:39.702693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1094/2183] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:34:39 2022] DELETE /identity/v3/users/d747a37351104754a8b2b475273732b4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:39.709016 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c244a59-4df3-4659-a1b9-466762240fdc None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:39.709306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c244a59-4df3-4659-a1b9-466762240fdc None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:39.710272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c244a59-4df3-4659-a1b9-466762240fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:39.732871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2c244a59-4df3-4659-a1b9-466762240fdc None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:39.734097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2c244a59-4df3-4659-a1b9-466762240fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:39.811965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1090/2184] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:39.823185 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:39.826104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:39.979397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f69430-440b-4eca-936f-50c6b919adf1 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:39.997379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b9e1b-2da5-4391-88c0-826c885e5435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:39.998019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b9e1b-2da5-4391-88c0-826c885e5435 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:39.998295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b9e1b-2da5-4391-88c0-826c885e5435 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:39.998565 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b9e1b-2da5-4391-88c0-826c885e5435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:40.024167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b9e1b-2da5-4391-88c0-826c885e5435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:40.025956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b9e1b-2da5-4391-88c0-826c885e5435 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:40.051151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1095/2185] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:34:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:34:40.334397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6558c6-e761-45d6-86f7-fa0957667b68 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:40.335442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6558c6-e761-45d6-86f7-fa0957667b68 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:40.335813 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6558c6-e761-45d6-86f7-fa0957667b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:40.364908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0d6558c6-e761-45d6-86f7-fa0957667b68 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:40.366737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0d6558c6-e761-45d6-86f7-fa0957667b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:40.453199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1091/2186] 10.222.0.55 () {58 vars in 967 bytes} [Fri Aug 19 16:34:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:40.460620 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b9e1b-2da5-4391-88c0-826c885e5435 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:40.467711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8f3b9e1b-2da5-4391-88c0-826c885e5435 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:40.472959 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6558c6-e761-45d6-86f7-fa0957667b68 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:40.475907 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0d6558c6-e761-45d6-86f7-fa0957667b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:40.521607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b9e1b-2da5-4391-88c0-826c885e5435 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:40.529167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:40.531038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:40.531496 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:40.532611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] PATH_INFO: `/v3/projects/8e7489c151da46479f9be2218e0a2f5b` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:40.538504 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e7489c151da46479f9be2218e0a2f5b)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:40.541688 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:40.608357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6558c6-e761-45d6-86f7-fa0957667b68 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:40.614272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627ddcf-4ad0-47d1-bb43-c7fcdb202245 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:40.615043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627ddcf-4ad0-47d1-bb43-c7fcdb202245 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:40.615466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627ddcf-4ad0-47d1-bb43-c7fcdb202245 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:40.616639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627ddcf-4ad0-47d1-bb43-c7fcdb202245 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:40.627543 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627ddcf-4ad0-47d1-bb43-c7fcdb202245 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:40.629564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627ddcf-4ad0-47d1-bb43-c7fcdb202245 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:40.645402 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1092/2187] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:34:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:34:40.650946 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:40.674136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e7489c151da46479f9be2218e0a2f5b', 'request_id': 'req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:40.684569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e7489c151da46479f9be2218e0a2f5b', 'request_id': 'req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:40.696232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e7489c151da46479f9be2218e0a2f5b', 'request_id': 'req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:40.730202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] The token cache is being invalidate because project 8e7489c151da46479f9be2218e0a2f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:34:40.730745 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:40.731699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:40.738138 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627ddcf-4ad0-47d1-bb43-c7fcdb202245 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:40.744538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1096/2188] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:34:40 2022] DELETE /identity/v3/projects/8e7489c151da46479f9be2218e0a2f5b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:40.751215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627ddcf-4ad0-47d1-bb43-c7fcdb202245 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:40.779813 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:40.846995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ec1b3-7554-4651-914b-5995a640ecde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:40.847583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ec1b3-7554-4651-914b-5995a640ecde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:40.847930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ec1b3-7554-4651-914b-5995a640ecde admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:40.848316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ec1b3-7554-4651-914b-5995a640ecde admin admin] PATH_INFO: `/v3/users/983b31dcab93430c904b0bbccaedc6b8` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:40.849797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ec1b3-7554-4651-914b-5995a640ecde admin admin] RBAC: Authorizing `identity:delete_user(user_id=983b31dcab93430c904b0bbccaedc6b8)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:40.851789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ec1b3-7554-4651-914b-5995a640ecde admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:40.917857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-bd0ec1b3-7554-4651-914b-5995a640ecde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '983b31dcab93430c904b0bbccaedc6b8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:40.919080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-bd0ec1b3-7554-4651-914b-5995a640ecde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '983b31dcab93430c904b0bbccaedc6b8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:40.924078 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-bd0ec1b3-7554-4651-914b-5995a640ecde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '983b31dcab93430c904b0bbccaedc6b8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:40.938130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-bd0ec1b3-7554-4651-914b-5995a640ecde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '983b31dcab93430c904b0bbccaedc6b8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:40.948126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-bd0ec1b3-7554-4651-914b-5995a640ecde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '983b31dcab93430c904b0bbccaedc6b8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:40.968437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1093/2189] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:34:40 2022] DELETE /identity/v3/users/983b31dcab93430c904b0bbccaedc6b8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:41.477972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:41.480362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:41.502173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba5b2a-c9bb-4cd8-870b-ffc445eca516 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:41.600038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3843e-945b-40c1-9095-303a9f0f383a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:41.601668 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3843e-945b-40c1-9095-303a9f0f383a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:41.602129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3843e-945b-40c1-9095-303a9f0f383a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:41.602563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3843e-945b-40c1-9095-303a9f0f383a admin admin] PATH_INFO: `/v3/projects/4f09b1e69b9a4ed58ff78ae6d1bbc87b` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:41.603747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3843e-945b-40c1-9095-303a9f0f383a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f09b1e69b9a4ed58ff78ae6d1bbc87b)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:41.605782 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3843e-945b-40c1-9095-303a9f0f383a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:41.670215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-abe3843e-945b-40c1-9095-303a9f0f383a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f09b1e69b9a4ed58ff78ae6d1bbc87b', 'request_id': 'req-abe3843e-945b-40c1-9095-303a9f0f383a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:41.677881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-abe3843e-945b-40c1-9095-303a9f0f383a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f09b1e69b9a4ed58ff78ae6d1bbc87b', 'request_id': 'req-abe3843e-945b-40c1-9095-303a9f0f383a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:41.685683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-abe3843e-945b-40c1-9095-303a9f0f383a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f09b1e69b9a4ed58ff78ae6d1bbc87b', 'request_id': 'req-abe3843e-945b-40c1-9095-303a9f0f383a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:41.703018 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-abe3843e-945b-40c1-9095-303a9f0f383a admin admin] The token cache is being invalidate because project 4f09b1e69b9a4ed58ff78ae6d1bbc87b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:34:41.703458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-abe3843e-945b-40c1-9095-303a9f0f383a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:41.704275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-abe3843e-945b-40c1-9095-303a9f0f383a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:41.706359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1097/2190] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:34:41 2022] DELETE /identity/v3/projects/4f09b1e69b9a4ed58ff78ae6d1bbc87b => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:41.712209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82b7b1-2143-4a11-b4e3-bd5736d9a70e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:41.712645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82b7b1-2143-4a11-b4e3-bd5736d9a70e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:41.712776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82b7b1-2143-4a11-b4e3-bd5736d9a70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:41.745354 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0e82b7b1-2143-4a11-b4e3-bd5736d9a70e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:41.746883 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0e82b7b1-2143-4a11-b4e3-bd5736d9a70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:41.842403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1094/2191] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:41.849396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d00ec-c446-49c8-817d-5cc3b94a384f None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:41.849883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d00ec-c446-49c8-817d-5cc3b94a384f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:41.850191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d00ec-c446-49c8-817d-5cc3b94a384f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:41.905368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f31d00ec-c446-49c8-817d-5cc3b94a384f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:41.906649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f31d00ec-c446-49c8-817d-5cc3b94a384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:41.926371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1098/2192] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:41.931700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82b7b1-2143-4a11-b4e3-bd5736d9a70e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:41.948435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82b7b1-2143-4a11-b4e3-bd5736d9a70e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:41.956417 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841b6e6-939e-47bb-b00a-1d72eb29b153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:41.957120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841b6e6-939e-47bb-b00a-1d72eb29b153 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:41.957438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841b6e6-939e-47bb-b00a-1d72eb29b153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:41.958373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841b6e6-939e-47bb-b00a-1d72eb29b153 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:41.959113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841b6e6-939e-47bb-b00a-1d72eb29b153 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:34:41.960484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841b6e6-939e-47bb-b00a-1d72eb29b153 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:41.962337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841b6e6-939e-47bb-b00a-1d72eb29b153 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:41.980173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1095/2193] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:34:41.985009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d00ec-c446-49c8-817d-5cc3b94a384f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:42.010747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d00ec-c446-49c8-817d-5cc3b94a384f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:42.015991 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16598e79-3d74-4abe-8488-b569215db200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:42.016637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16598e79-3d74-4abe-8488-b569215db200 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:42.017022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16598e79-3d74-4abe-8488-b569215db200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:42.017299 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16598e79-3d74-4abe-8488-b569215db200 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:42.017956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16598e79-3d74-4abe-8488-b569215db200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:42.019871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16598e79-3d74-4abe-8488-b569215db200 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:42.075165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1099/2194] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:34:41 2022] POST /identity/v3/projects => generated 381 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:42.076531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841b6e6-939e-47bb-b00a-1d72eb29b153 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:42.093873 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841b6e6-939e-47bb-b00a-1d72eb29b153 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:42.126165 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:42.167633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba27f8-9122-4a83-bded-fc7a1ccd25b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:42.168563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba27f8-9122-4a83-bded-fc7a1ccd25b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:42.169122 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba27f8-9122-4a83-bded-fc7a1ccd25b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:42.169718 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba27f8-9122-4a83-bded-fc7a1ccd25b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:42.170561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba27f8-9122-4a83-bded-fc7a1ccd25b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:42.173007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba27f8-9122-4a83-bded-fc7a1ccd25b0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:42.202561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1096/2195] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:34:42 2022] POST /identity/v3/users => generated 291 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:42.204584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16598e79-3d74-4abe-8488-b569215db200 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:42.216925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16598e79-3d74-4abe-8488-b569215db200 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:42.222844 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab9c52-7656-4b27-9c31-f67d1da9f194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:42.223451 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab9c52-7656-4b27-9c31-f67d1da9f194 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:42.223729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab9c52-7656-4b27-9c31-f67d1da9f194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:42.224073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab9c52-7656-4b27-9c31-f67d1da9f194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:42.225345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab9c52-7656-4b27-9c31-f67d1da9f194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:42.228429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab9c52-7656-4b27-9c31-f67d1da9f194 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:42.244013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1100/2196] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:42.247458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba27f8-9122-4a83-bded-fc7a1ccd25b0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:42.263405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba27f8-9122-4a83-bded-fc7a1ccd25b0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:42.269003 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a621c8c-0976-4c5e-b5de-c5962a34d4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:42.269704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a621c8c-0976-4c5e-b5de-c5962a34d4fd admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:42.270080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a621c8c-0976-4c5e-b5de-c5962a34d4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:42.270447 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a621c8c-0976-4c5e-b5de-c5962a34d4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:42.271222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a621c8c-0976-4c5e-b5de-c5962a34d4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:42.273138 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a621c8c-0976-4c5e-b5de-c5962a34d4fd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:42.286804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1097/2197] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:42.291042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab9c52-7656-4b27-9c31-f67d1da9f194 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:42.307255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab9c52-7656-4b27-9c31-f67d1da9f194 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:42.312493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6e6bd-a397-4930-ac5e-5b4e9877a0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:42.313312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6e6bd-a397-4930-ac5e-5b4e9877a0fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:42.313689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6e6bd-a397-4930-ac5e-5b4e9877a0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:42.316319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6e6bd-a397-4930-ac5e-5b4e9877a0fb admin admin] PATH_INFO: `/v3/projects/908065bedcb74edcaeadb3e3fc0fed53/users/b61d6c657c4b4eb9aab2477bc6811e9f/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:42.339203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6e6bd-a397-4930-ac5e-5b4e9877a0fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=908065bedcb74edcaeadb3e3fc0fed53, user_id=b61d6c657c4b4eb9aab2477bc6811e9f, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:42.342160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6e6bd-a397-4930-ac5e-5b4e9877a0fb admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:42.346181 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:42.375090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dba7f-51a3-477a-88fb-d5d457c36930 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:42.375509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dba7f-51a3-477a-88fb-d5d457c36930 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:42.375851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dba7f-51a3-477a-88fb-d5d457c36930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:42.379374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1101/2198] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:42 2022] PUT /identity/v3/projects/908065bedcb74edcaeadb3e3fc0fed53/users/b61d6c657c4b4eb9aab2477bc6811e9f/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:34:42.407997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-771dba7f-51a3-477a-88fb-d5d457c36930 None None] MFA Rules not processed for user `b61d6c657c4b4eb9aab2477bc6811e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:42.411189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-771dba7f-51a3-477a-88fb-d5d457c36930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:42.536202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1098/2199] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:42.541033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6e6bd-a397-4930-ac5e-5b4e9877a0fb admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:42.680862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6e6bd-a397-4930-ac5e-5b4e9877a0fb admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:42.693956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023c170-7c86-47f9-b6b4-e8915b482b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:42.694597 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023c170-7c86-47f9-b6b4-e8915b482b75 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:42.694878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023c170-7c86-47f9-b6b4-e8915b482b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:42.695201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023c170-7c86-47f9-b6b4-e8915b482b75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:42.695847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023c170-7c86-47f9-b6b4-e8915b482b75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:42.698055 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023c170-7c86-47f9-b6b4-e8915b482b75 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:42.754268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1102/2200] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:34:42 2022] POST /identity/v3/projects => generated 379 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:42.766105 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dba7f-51a3-477a-88fb-d5d457c36930 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:42.788739 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dba7f-51a3-477a-88fb-d5d457c36930 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:42.901058 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be72b1b-fdf0-46ae-bf63-67ebbc1beaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:42.902113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be72b1b-fdf0-46ae-bf63-67ebbc1beaaf admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:42.902610 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be72b1b-fdf0-46ae-bf63-67ebbc1beaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:42.903112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be72b1b-fdf0-46ae-bf63-67ebbc1beaaf admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:42.904100 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be72b1b-fdf0-46ae-bf63-67ebbc1beaaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:42.911042 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be72b1b-fdf0-46ae-bf63-67ebbc1beaaf admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:42.956104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023c170-7c86-47f9-b6b4-e8915b482b75 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:42.959312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1099/2201] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:34:42 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:42.974288 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023c170-7c86-47f9-b6b4-e8915b482b75 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:42.983874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638229f-ee0e-4659-bf7e-799365440ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:42.984936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638229f-ee0e-4659-bf7e-799365440ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:42.985448 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638229f-ee0e-4659-bf7e-799365440ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:42.985973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638229f-ee0e-4659-bf7e-799365440ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:42.987073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638229f-ee0e-4659-bf7e-799365440ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:42.989226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638229f-ee0e-4659-bf7e-799365440ad5 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:43.012652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be72b1b-fdf0-46ae-bf63-67ebbc1beaaf admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:43.021567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1103/2202] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:43.026569 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be72b1b-fdf0-46ae-bf63-67ebbc1beaaf admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:43.032664 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49ad71-a990-4eed-930c-f0334cad546b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:43.033832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49ad71-a990-4eed-930c-f0334cad546b admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:43.034343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49ad71-a990-4eed-930c-f0334cad546b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:43.034830 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49ad71-a990-4eed-930c-f0334cad546b admin admin] PATH_INFO: `/v3/projects/31ce5436b1e742ec8119dd2fb9c312a4/users/69e806adecde445d8f7b2b180cc5f9a9/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:43.057246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49ad71-a990-4eed-930c-f0334cad546b admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ce5436b1e742ec8119dd2fb9c312a4, user_id=69e806adecde445d8f7b2b180cc5f9a9, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:43.061146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49ad71-a990-4eed-930c-f0334cad546b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:43.076408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:43.094859 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1100/2203] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:43 2022] PUT /identity/v3/projects/31ce5436b1e742ec8119dd2fb9c312a4/users/69e806adecde445d8f7b2b180cc5f9a9/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:43.095012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638229f-ee0e-4659-bf7e-799365440ad5 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:43.117032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638229f-ee0e-4659-bf7e-799365440ad5 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:43.254471 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb19c4-d8af-4c0b-8040-698a57cfd362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:43.255008 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb19c4-d8af-4c0b-8040-698a57cfd362 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:43.255293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb19c4-d8af-4c0b-8040-698a57cfd362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:43.255618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb19c4-d8af-4c0b-8040-698a57cfd362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:43.261437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb19c4-d8af-4c0b-8040-698a57cfd362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:43.266112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49ad71-a990-4eed-930c-f0334cad546b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:43.267711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ca49ad71-a990-4eed-930c-f0334cad546b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:43.278557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb19c4-d8af-4c0b-8040-698a57cfd362 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:43.294272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49ad71-a990-4eed-930c-f0334cad546b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:43.301165 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:43.301818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:43.302300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1104/2204] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:34:43.308636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:43.309070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] PATH_INFO: `/v3/users/7536a92033384383bd625b6c969ebd38` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:43.309293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb19c4-d8af-4c0b-8040-698a57cfd362 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:43.310815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7536a92033384383bd625b6c969ebd38)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:43.312700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:43.323727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb19c4-d8af-4c0b-8040-698a57cfd362 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:43.332600 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2136430-e0ab-4241-96ff-77bf82ef158f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:43.333373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2136430-e0ab-4241-96ff-77bf82ef158f admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:43.333722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2136430-e0ab-4241-96ff-77bf82ef158f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:43.334063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2136430-e0ab-4241-96ff-77bf82ef158f admin admin] PATH_INFO: `/v3/projects/31ce5436b1e742ec8119dd2fb9c312a4/users/69e806adecde445d8f7b2b180cc5f9a9/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:43.337124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2136430-e0ab-4241-96ff-77bf82ef158f admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ce5436b1e742ec8119dd2fb9c312a4, user_id=69e806adecde445d8f7b2b180cc5f9a9, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:43.340061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2136430-e0ab-4241-96ff-77bf82ef158f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:43.353395 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1105/2205] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:43 2022] PUT /identity/v3/projects/31ce5436b1e742ec8119dd2fb9c312a4/users/69e806adecde445d8f7b2b180cc5f9a9/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:43.372340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f0e7c-ce24-415f-bf6d-81a94fd647a4 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:43.372630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f0e7c-ce24-415f-bf6d-81a94fd647a4 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:43.372946 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f0e7c-ce24-415f-bf6d-81a94fd647a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:43.438217 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7536a92033384383bd625b6c969ebd38'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:43.438729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-260f0e7c-ce24-415f-bf6d-81a94fd647a4 None None] MFA Rules not processed for user `69e806adecde445d8f7b2b180cc5f9a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:43.439488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7536a92033384383bd625b6c969ebd38'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:43.440147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7536a92033384383bd625b6c969ebd38'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:43.440670 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-260f0e7c-ce24-415f-bf6d-81a94fd647a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:43.455325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7536a92033384383bd625b6c969ebd38'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:43.463042 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7536a92033384383bd625b6c969ebd38'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:43.498372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1101/2206] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:34:43 2022] DELETE /identity/v3/users/7536a92033384383bd625b6c969ebd38 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:43.507295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:43.556976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1106/2207] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:43.570635 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:43.572121 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:43.668525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe891d-3d7d-4c25-9fba-8f0e596ce0c0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:43.673633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930586f-f4ec-4f9c-99d7-0134b801e8f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:43.674262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930586f-f4ec-4f9c-99d7-0134b801e8f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:43.674544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930586f-f4ec-4f9c-99d7-0134b801e8f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:43.674818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930586f-f4ec-4f9c-99d7-0134b801e8f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:43.816665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930586f-f4ec-4f9c-99d7-0134b801e8f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:43.820348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930586f-f4ec-4f9c-99d7-0134b801e8f3 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:43.847607 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1102/2208] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:34:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:34:44.259442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f0e7c-ce24-415f-bf6d-81a94fd647a4 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:44.261475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-260f0e7c-ce24-415f-bf6d-81a94fd647a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:44.374407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f0e7c-ce24-415f-bf6d-81a94fd647a4 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:44.481301 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:44.482425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:44.482773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:44.483115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] PATH_INFO: `/v3/projects/a549f3d5fb9443409faa06cf7b00d8ca` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:44.485195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a549f3d5fb9443409faa06cf7b00d8ca)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:44.500569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:44.592637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930586f-f4ec-4f9c-99d7-0134b801e8f3 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:44.612491 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930586f-f4ec-4f9c-99d7-0134b801e8f3 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:44.619214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a549f3d5fb9443409faa06cf7b00d8ca', 'request_id': 'req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:44.626214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a549f3d5fb9443409faa06cf7b00d8ca', 'request_id': 'req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:44.634114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a549f3d5fb9443409faa06cf7b00d8ca', 'request_id': 'req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:44.658074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] The token cache is being invalidate because project a549f3d5fb9443409faa06cf7b00d8ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:34:44.658367 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:44.659490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:44.662559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1107/2209] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:34:44 2022] DELETE /identity/v3/projects/a549f3d5fb9443409faa06cf7b00d8ca => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:44.715825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:44.884155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c7fe3-a71f-4b0d-ab27-e780d88afd70 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e806adecde445d8f7b2b180cc5f9a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31ce5436b1e742ec8119dd2fb9c312a4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:44.887341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c7fe3-a71f-4b0d-ab27-e780d88afd70 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:44.887927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c7fe3-a71f-4b0d-ab27-e780d88afd70 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:44.888388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c7fe3-a71f-4b0d-ab27-e780d88afd70 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:44.889313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c7fe3-a71f-4b0d-ab27-e780d88afd70 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:34:44.889778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c7fe3-a71f-4b0d-ab27-e780d88afd70 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:44.893402 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c7fe3-a71f-4b0d-ab27-e780d88afd70 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:44.922724 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1103/2210] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:34:44.927093 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:44.929292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:44.973211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4968e0-3d82-4ba3-9164-a0cbf0ca5c10 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:45.013033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc7e1b-c4d9-49c6-b2bc-9b8435f9beee tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e806adecde445d8f7b2b180cc5f9a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31ce5436b1e742ec8119dd2fb9c312a4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:45.015663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc7e1b-c4d9-49c6-b2bc-9b8435f9beee tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:45.016067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc7e1b-c4d9-49c6-b2bc-9b8435f9beee tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:45.018860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc7e1b-c4d9-49c6-b2bc-9b8435f9beee tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:45.020151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc7e1b-c4d9-49c6-b2bc-9b8435f9beee tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:45.025378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc7e1b-c4d9-49c6-b2bc-9b8435f9beee tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:45.069596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:45.100578 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1108/2211] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:34:44 2022] POST /identity/v3/projects => generated 353 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:45.104292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c7fe3-a71f-4b0d-ab27-e780d88afd70 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:45.130327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c7fe3-a71f-4b0d-ab27-e780d88afd70 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:45.235508 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d1ea8-8ea3-4efa-901d-4b30b83d070f tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e806adecde445d8f7b2b180cc5f9a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31ce5436b1e742ec8119dd2fb9c312a4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:45.236457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d1ea8-8ea3-4efa-901d-4b30b83d070f tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:45.237004 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d1ea8-8ea3-4efa-901d-4b30b83d070f tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:45.237591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d1ea8-8ea3-4efa-901d-4b30b83d070f tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:45.238600 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d1ea8-8ea3-4efa-901d-4b30b83d070f tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:34:45.239091 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d1ea8-8ea3-4efa-901d-4b30b83d070f tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:45.241237 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d1ea8-8ea3-4efa-901d-4b30b83d070f tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:45.255479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1104/2212] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:34:45.300189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc7e1b-c4d9-49c6-b2bc-9b8435f9beee tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:45.312119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc7e1b-c4d9-49c6-b2bc-9b8435f9beee tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:45.317955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f1a15-c722-4a77-a8bf-2438e3214349 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e806adecde445d8f7b2b180cc5f9a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31ce5436b1e742ec8119dd2fb9c312a4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:45.319579 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f1a15-c722-4a77-a8bf-2438e3214349 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:45.319934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f1a15-c722-4a77-a8bf-2438e3214349 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:45.320140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f1a15-c722-4a77-a8bf-2438e3214349 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] PATH_INFO: `/v3/projects/9f8eb304628a4b84bc65c78c7770a22b` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:45.322937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f1a15-c722-4a77-a8bf-2438e3214349 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9f8eb304628a4b84bc65c78c7770a22b)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:45.325389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f1a15-c722-4a77-a8bf-2438e3214349 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:45.389843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d82f1a15-c722-4a77-a8bf-2438e3214349 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f8eb304628a4b84bc65c78c7770a22b', 'request_id': 'req-d82f1a15-c722-4a77-a8bf-2438e3214349'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:45.397179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d82f1a15-c722-4a77-a8bf-2438e3214349 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f8eb304628a4b84bc65c78c7770a22b', 'request_id': 'req-d82f1a15-c722-4a77-a8bf-2438e3214349'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:45.405377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d82f1a15-c722-4a77-a8bf-2438e3214349 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f8eb304628a4b84bc65c78c7770a22b', 'request_id': 'req-d82f1a15-c722-4a77-a8bf-2438e3214349'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:45.429271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d82f1a15-c722-4a77-a8bf-2438e3214349 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] The token cache is being invalidate because project 9f8eb304628a4b84bc65c78c7770a22b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:34:45.429918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d82f1a15-c722-4a77-a8bf-2438e3214349 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:45.430867 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d82f1a15-c722-4a77-a8bf-2438e3214349 tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:45.433108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1109/2213] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:34:45 2022] DELETE /identity/v3/projects/9f8eb304628a4b84bc65c78c7770a22b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:45.832015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d1ea8-8ea3-4efa-901d-4b30b83d070f tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:45.837916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fd5d1ea8-8ea3-4efa-901d-4b30b83d070f tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:45.988254 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d1ea8-8ea3-4efa-901d-4b30b83d070f tempest-VolumeQuotaClassesTest-588007503 tempest-VolumeQuotaClassesTest-588007503-project-admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:46.097721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:46.110912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:46.113731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:46.114637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:46.114637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] PATH_INFO: `/v3/users/b61d6c657c4b4eb9aab2477bc6811e9f` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:46.115874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b61d6c657c4b4eb9aab2477bc6811e9f)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:46.119944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:46.223960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b61d6c657c4b4eb9aab2477bc6811e9f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:46.225651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b61d6c657c4b4eb9aab2477bc6811e9f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:46.226627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b61d6c657c4b4eb9aab2477bc6811e9f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:46.257596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b61d6c657c4b4eb9aab2477bc6811e9f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:46.273848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b61d6c657c4b4eb9aab2477bc6811e9f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:46.312046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1105/2214] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:34:45 2022] DELETE /identity/v3/users/b61d6c657c4b4eb9aab2477bc6811e9f => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:46.314452 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c2e4f-9395-479d-ac63-1ebbbb9ccda0 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:46.315454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c2e4f-9395-479d-ac63-1ebbbb9ccda0 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:46.315909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c2e4f-9395-479d-ac63-1ebbbb9ccda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:46.360143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-af4c2e4f-9395-479d-ac63-1ebbbb9ccda0 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:46.361940 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-af4c2e4f-9395-479d-ac63-1ebbbb9ccda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:46.445278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1110/2215] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:34:46.460841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:46.462480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:46.589364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddb57c-af98-440e-a6f8-55620f836df5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:46.594030 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02278535-f538-404d-a46b-49490f93a788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:46.594665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02278535-f538-404d-a46b-49490f93a788 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:46.594950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02278535-f538-404d-a46b-49490f93a788 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:46.595375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02278535-f538-404d-a46b-49490f93a788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:46.611980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02278535-f538-404d-a46b-49490f93a788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:46.612678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02278535-f538-404d-a46b-49490f93a788 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:46.645503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1106/2216] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:34:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:34:47.100931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c2e4f-9395-479d-ac63-1ebbbb9ccda0 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:47.103290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-af4c2e4f-9395-479d-ac63-1ebbbb9ccda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:47.129255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c2e4f-9395-479d-ac63-1ebbbb9ccda0 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:47.156382 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:47.156382 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:47.156382 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:47.156382 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] PATH_INFO: `/v3/projects/908065bedcb74edcaeadb3e3fc0fed53` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:47.157334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=908065bedcb74edcaeadb3e3fc0fed53)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:47.159575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:47.246101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '908065bedcb74edcaeadb3e3fc0fed53', 'request_id': 'req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:47.253186 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '908065bedcb74edcaeadb3e3fc0fed53', 'request_id': 'req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:47.261193 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '908065bedcb74edcaeadb3e3fc0fed53', 'request_id': 'req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:47.276521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:INSERT=3,keystone:SELECT=83,keystone:DELETE=14 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:47.283138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] The token cache is being invalidate because project 908065bedcb74edcaeadb3e3fc0fed53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:34:47.283642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:47.284797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:47.286935 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1111/2217] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:34:47 2022] DELETE /identity/v3/projects/908065bedcb74edcaeadb3e3fc0fed53 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:47.295084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02278535-f538-404d-a46b-49490f93a788 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:47.312451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02278535-f538-404d-a46b-49490f93a788 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:47.398361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa97a-4e2e-4379-ae47-9bb0b6ac2748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:47.399029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa97a-4e2e-4379-ae47-9bb0b6ac2748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:47.402013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa97a-4e2e-4379-ae47-9bb0b6ac2748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:47.402318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa97a-4e2e-4379-ae47-9bb0b6ac2748 admin admin] PATH_INFO: `/v3/users/69e806adecde445d8f7b2b180cc5f9a9` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:47.404246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54aa97a-4e2e-4379-ae47-9bb0b6ac2748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e806adecde445d8f7b2b180cc5f9a9)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:47.409079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54aa97a-4e2e-4379-ae47-9bb0b6ac2748 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:47.464104 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:47.504431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d54aa97a-4e2e-4379-ae47-9bb0b6ac2748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e806adecde445d8f7b2b180cc5f9a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:47.505637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d54aa97a-4e2e-4379-ae47-9bb0b6ac2748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e806adecde445d8f7b2b180cc5f9a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:47.506836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d54aa97a-4e2e-4379-ae47-9bb0b6ac2748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e806adecde445d8f7b2b180cc5f9a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:47.528210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d54aa97a-4e2e-4379-ae47-9bb0b6ac2748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e806adecde445d8f7b2b180cc5f9a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:47.536304 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d54aa97a-4e2e-4379-ae47-9bb0b6ac2748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e806adecde445d8f7b2b180cc5f9a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:47.562982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1107/2218] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:34:47 2022] DELETE /identity/v3/users/69e806adecde445d8f7b2b180cc5f9a9 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:47.917822 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:47.919294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:47.946852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb0ed-a522-43c6-b5ab-2aedd7bc0b0c admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:48.048381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:48.050016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:48.050489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:48.051818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705 admin admin] PATH_INFO: `/v3/projects/31ce5436b1e742ec8119dd2fb9c312a4` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:48.059996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ce5436b1e742ec8119dd2fb9c312a4)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:48.062521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:48.169607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ce5436b1e742ec8119dd2fb9c312a4', 'request_id': 'req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:48.190870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ce5436b1e742ec8119dd2fb9c312a4', 'request_id': 'req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:48.199247 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ce5436b1e742ec8119dd2fb9c312a4', 'request_id': 'req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:48.224492 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705 admin admin] The token cache is being invalidate because project 31ce5436b1e742ec8119dd2fb9c312a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:34:48.225418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:48.226327 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b60e9e61-bbc1-4c1a-929e-7c5c5b480705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:48.227904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1112/2219] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:34:47 2022] DELETE /identity/v3/projects/31ce5436b1e742ec8119dd2fb9c312a4 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:48.232879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6022a3-b65e-40bf-b77a-7ed625f894a8 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:48.233428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6022a3-b65e-40bf-b77a-7ed625f894a8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:48.233927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6022a3-b65e-40bf-b77a-7ed625f894a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:48.259774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9e6022a3-b65e-40bf-b77a-7ed625f894a8 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:48.261055 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9e6022a3-b65e-40bf-b77a-7ed625f894a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:48.349599 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1108/2220] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:48.360591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5d012-ebae-4416-b16b-be2bacb3eae4 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:48.361202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5d012-ebae-4416-b16b-be2bacb3eae4 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:48.361672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5d012-ebae-4416-b16b-be2bacb3eae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:48.388924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-09a5d012-ebae-4416-b16b-be2bacb3eae4 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:48.390503 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-09a5d012-ebae-4416-b16b-be2bacb3eae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:48.400641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1113/2221] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:48.405026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6022a3-b65e-40bf-b77a-7ed625f894a8 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:48.418051 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6022a3-b65e-40bf-b77a-7ed625f894a8 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:48.423290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f7b80-cbfc-4ea8-8803-1570e89135a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:48.424041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f7b80-cbfc-4ea8-8803-1570e89135a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:48.424628 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f7b80-cbfc-4ea8-8803-1570e89135a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:48.425014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f7b80-cbfc-4ea8-8803-1570e89135a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:48.425832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f7b80-cbfc-4ea8-8803-1570e89135a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:34:48.426301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f7b80-cbfc-4ea8-8803-1570e89135a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:48.430398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f7b80-cbfc-4ea8-8803-1570e89135a9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:48.454250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1109/2222] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:34:48.461770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5d012-ebae-4416-b16b-be2bacb3eae4 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:48.471881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5d012-ebae-4416-b16b-be2bacb3eae4 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:48.477531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e041e-0d7d-499c-aac8-cd7fc3136497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:48.478418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e041e-0d7d-499c-aac8-cd7fc3136497 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:48.478734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e041e-0d7d-499c-aac8-cd7fc3136497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:48.479012 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e041e-0d7d-499c-aac8-cd7fc3136497 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:48.479634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e041e-0d7d-499c-aac8-cd7fc3136497 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:48.481760 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e041e-0d7d-499c-aac8-cd7fc3136497 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:48.519363 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1114/2223] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:34:48 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:48.523697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f7b80-cbfc-4ea8-8803-1570e89135a9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:48.534132 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f7b80-cbfc-4ea8-8803-1570e89135a9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:48.602889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d714b0f-4124-4402-8b2f-be9a88dadb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:48.603622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d714b0f-4124-4402-8b2f-be9a88dadb2c admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:48.603903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d714b0f-4124-4402-8b2f-be9a88dadb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:48.604173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d714b0f-4124-4402-8b2f-be9a88dadb2c admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:48.604821 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d714b0f-4124-4402-8b2f-be9a88dadb2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:48.606916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d714b0f-4124-4402-8b2f-be9a88dadb2c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:48.635341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1110/2224] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:34:48 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:48.636697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e041e-0d7d-499c-aac8-cd7fc3136497 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:48.660834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e041e-0d7d-499c-aac8-cd7fc3136497 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:48.667473 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b32968-e922-4979-b806-b415b5dd3022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:48.668223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b32968-e922-4979-b806-b415b5dd3022 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:48.668512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b32968-e922-4979-b806-b415b5dd3022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:48.668887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b32968-e922-4979-b806-b415b5dd3022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:48.669541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b32968-e922-4979-b806-b415b5dd3022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:48.671138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b32968-e922-4979-b806-b415b5dd3022 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:48.685590 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1115/2225] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:48.686997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d714b0f-4124-4402-8b2f-be9a88dadb2c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:48.699321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d714b0f-4124-4402-8b2f-be9a88dadb2c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:48.704913 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8c203-1cb7-4557-9a3b-92fd6050c9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:48.705594 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8c203-1cb7-4557-9a3b-92fd6050c9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:48.705942 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8c203-1cb7-4557-9a3b-92fd6050c9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:48.706282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8c203-1cb7-4557-9a3b-92fd6050c9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:48.706987 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8c203-1cb7-4557-9a3b-92fd6050c9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:48.708686 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8c203-1cb7-4557-9a3b-92fd6050c9a6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:48.725871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1111/2226] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:48.730255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b32968-e922-4979-b806-b415b5dd3022 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:48.743301 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b32968-e922-4979-b806-b415b5dd3022 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:48.748077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be2fcb-7371-436c-a725-4c1c7bf7aa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:48.748771 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be2fcb-7371-436c-a725-4c1c7bf7aa53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:48.749099 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be2fcb-7371-436c-a725-4c1c7bf7aa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:48.749373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be2fcb-7371-436c-a725-4c1c7bf7aa53 admin admin] PATH_INFO: `/v3/projects/3fd2bc8a84e84fc08bd80d78f2c5eca9/users/821dc921753e4232b4e3136c1ff876b3/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:48.767556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be2fcb-7371-436c-a725-4c1c7bf7aa53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd2bc8a84e84fc08bd80d78f2c5eca9, user_id=821dc921753e4232b4e3136c1ff876b3, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:48.770744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be2fcb-7371-436c-a725-4c1c7bf7aa53 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:48.786711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1116/2227] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:48 2022] PUT /identity/v3/projects/3fd2bc8a84e84fc08bd80d78f2c5eca9/users/821dc921753e4232b4e3136c1ff876b3/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:48.797953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69527e48-0aa3-43fa-a032-cd6158e85714 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:48.798249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69527e48-0aa3-43fa-a032-cd6158e85714 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:48.798564 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69527e48-0aa3-43fa-a032-cd6158e85714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:48.807950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:48.837743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-69527e48-0aa3-43fa-a032-cd6158e85714 None None] MFA Rules not processed for user `821dc921753e4232b4e3136c1ff876b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:48.839910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-69527e48-0aa3-43fa-a032-cd6158e85714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:48.918200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1112/2228] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:48.922539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be2fcb-7371-436c-a725-4c1c7bf7aa53 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:48.971792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:49.017493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be2fcb-7371-436c-a725-4c1c7bf7aa53 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:49.024390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96621ce0-bee5-4f16-bd7a-25067bcdf2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:49.024798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96621ce0-bee5-4f16-bd7a-25067bcdf2ec admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:49.025095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96621ce0-bee5-4f16-bd7a-25067bcdf2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:49.025566 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96621ce0-bee5-4f16-bd7a-25067bcdf2ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:49.026702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96621ce0-bee5-4f16-bd7a-25067bcdf2ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:49.028718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96621ce0-bee5-4f16-bd7a-25067bcdf2ec admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:49.065398 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1117/2229] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:34:48 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:49.069445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69527e48-0aa3-43fa-a032-cd6158e85714 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:49.082421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69527e48-0aa3-43fa-a032-cd6158e85714 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:49.153068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e11922-3135-47ec-a3ff-f43ae3983af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:49.153697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e11922-3135-47ec-a3ff-f43ae3983af8 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:49.153972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e11922-3135-47ec-a3ff-f43ae3983af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:49.154423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e11922-3135-47ec-a3ff-f43ae3983af8 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:49.155034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e11922-3135-47ec-a3ff-f43ae3983af8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:49.156976 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e11922-3135-47ec-a3ff-f43ae3983af8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:49.199663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1113/2230] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:34:49 2022] POST /identity/v3/users => generated 297 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:34:49.203472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96621ce0-bee5-4f16-bd7a-25067bcdf2ec admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:49.215770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96621ce0-bee5-4f16-bd7a-25067bcdf2ec admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:49.223050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fb924-cff6-447b-89e1-7a547758a69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:49.223805 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fb924-cff6-447b-89e1-7a547758a69c admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:49.224392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fb924-cff6-447b-89e1-7a547758a69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:49.224879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fb924-cff6-447b-89e1-7a547758a69c admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:49.225827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fb924-cff6-447b-89e1-7a547758a69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:49.227542 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fb924-cff6-447b-89e1-7a547758a69c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:49.242828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1118/2231] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:34:49.246808 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e11922-3135-47ec-a3ff-f43ae3983af8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:49.260085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e11922-3135-47ec-a3ff-f43ae3983af8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:49.267423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fca66a-f7d6-43fb-ac33-6d51ed8df6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:49.268220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fca66a-f7d6-43fb-ac33-6d51ed8df6c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:49.268438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fca66a-f7d6-43fb-ac33-6d51ed8df6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:49.268734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fca66a-f7d6-43fb-ac33-6d51ed8df6c7 admin admin] PATH_INFO: `/v3/projects/075a043bf9cb4c4cb854cd710db906ef/users/5d3b8fd1dbe54b0aa593c249d9f8a731/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:49.293956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fca66a-f7d6-43fb-ac33-6d51ed8df6c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=075a043bf9cb4c4cb854cd710db906ef, user_id=5d3b8fd1dbe54b0aa593c249d9f8a731, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:49.305144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fca66a-f7d6-43fb-ac33-6d51ed8df6c7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:49.322798 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1114/2232] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:49 2022] PUT /identity/v3/projects/075a043bf9cb4c4cb854cd710db906ef/users/5d3b8fd1dbe54b0aa593c249d9f8a731/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:49.325978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fb924-cff6-447b-89e1-7a547758a69c admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:49.337887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fb924-cff6-447b-89e1-7a547758a69c admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:49.439548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333280d-81eb-40bc-9a56-ea0bc9103fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:49.440062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333280d-81eb-40bc-9a56-ea0bc9103fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:49.440345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333280d-81eb-40bc-9a56-ea0bc9103fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:49.440616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333280d-81eb-40bc-9a56-ea0bc9103fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:49.441283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333280d-81eb-40bc-9a56-ea0bc9103fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:49.442868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333280d-81eb-40bc-9a56-ea0bc9103fd8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:49.457166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1119/2233] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:49.460097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fca66a-f7d6-43fb-ac33-6d51ed8df6c7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:49.471711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fca66a-f7d6-43fb-ac33-6d51ed8df6c7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:49.480349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6964db-004a-4219-8afa-d00270e20b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:49.481154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6964db-004a-4219-8afa-d00270e20b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:49.481500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6964db-004a-4219-8afa-d00270e20b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:49.481849 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6964db-004a-4219-8afa-d00270e20b2c admin admin] PATH_INFO: `/v3/projects/075a043bf9cb4c4cb854cd710db906ef/users/5d3b8fd1dbe54b0aa593c249d9f8a731/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:49.495733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6964db-004a-4219-8afa-d00270e20b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=075a043bf9cb4c4cb854cd710db906ef, user_id=5d3b8fd1dbe54b0aa593c249d9f8a731, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:49.498712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6964db-004a-4219-8afa-d00270e20b2c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:49.537255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1115/2234] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:49 2022] PUT /identity/v3/projects/075a043bf9cb4c4cb854cd710db906ef/users/5d3b8fd1dbe54b0aa593c249d9f8a731/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:49.539648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3781341-784d-4108-940e-75f113a149e2 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:49.540157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3781341-784d-4108-940e-75f113a149e2 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:49.540645 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3781341-784d-4108-940e-75f113a149e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:49.633852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f3781341-784d-4108-940e-75f113a149e2 None None] MFA Rules not processed for user `5d3b8fd1dbe54b0aa593c249d9f8a731`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:49.635396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f3781341-784d-4108-940e-75f113a149e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:49.763551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1120/2235] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:49.770758 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6964db-004a-4219-8afa-d00270e20b2c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:49.773245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fe6964db-004a-4219-8afa-d00270e20b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:49.849720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:49.857043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6964db-004a-4219-8afa-d00270e20b2c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:49.867917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812acbc7-9f73-47f7-9423-ec10e6947631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:49.869379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812acbc7-9f73-47f7-9423-ec10e6947631 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:49.869776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812acbc7-9f73-47f7-9423-ec10e6947631 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:49.870423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812acbc7-9f73-47f7-9423-ec10e6947631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:49.882422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812acbc7-9f73-47f7-9423-ec10e6947631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:49.884331 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812acbc7-9f73-47f7-9423-ec10e6947631 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:49.897717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1116/2236] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:34:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:34:50.018467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3781341-784d-4108-940e-75f113a149e2 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:50.031058 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3781341-784d-4108-940e-75f113a149e2 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:50.034795 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab14eaf-192e-472a-9d57-6207bf311382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:50.035513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab14eaf-192e-472a-9d57-6207bf311382 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:50.035800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab14eaf-192e-472a-9d57-6207bf311382 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:50.036076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab14eaf-192e-472a-9d57-6207bf311382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:50.062144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:50.120048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab14eaf-192e-472a-9d57-6207bf311382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:50.121571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab14eaf-192e-472a-9d57-6207bf311382 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:50.134781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1121/2237] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:34:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:34:56.040527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812acbc7-9f73-47f7-9423-ec10e6947631 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:56.117261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812acbc7-9f73-47f7-9423-ec10e6947631 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:56.121865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:56.122495 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:56.122778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:56.123056 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] PATH_INFO: `/v3/users/821dc921753e4232b4e3136c1ff876b3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:56.124182 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=821dc921753e4232b4e3136c1ff876b3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:56.126146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:56.172337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '821dc921753e4232b4e3136c1ff876b3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.173046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '821dc921753e4232b4e3136c1ff876b3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.173664 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '821dc921753e4232b4e3136c1ff876b3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.186649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '821dc921753e4232b4e3136c1ff876b3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.193917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '821dc921753e4232b4e3136c1ff876b3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.207827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1117/2238] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:34:56 2022] DELETE /identity/v3/users/821dc921753e4232b4e3136c1ff876b3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:56.212589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5ab99-94eb-40cd-9260-6b55ebdeaf57 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:56.213105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5ab99-94eb-40cd-9260-6b55ebdeaf57 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:56.213212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5ab99-94eb-40cd-9260-6b55ebdeaf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:56.232569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-a2f5ab99-94eb-40cd-9260-6b55ebdeaf57 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:56.233804 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a2f5ab99-94eb-40cd-9260-6b55ebdeaf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:56.296888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1122/2239] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:56.307418 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:56.308846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:56.366277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff55698-1e07-4ea1-8a07-e9481fe29adb admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:56.369982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcb096-22b9-45b0-815c-9b94458d46c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:56.370679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcb096-22b9-45b0-815c-9b94458d46c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:56.370961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcb096-22b9-45b0-815c-9b94458d46c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:56.371254 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcb096-22b9-45b0-815c-9b94458d46c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:56.380216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcb096-22b9-45b0-815c-9b94458d46c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:56.381786 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcb096-22b9-45b0-815c-9b94458d46c1 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:56.392015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1118/2240] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:34:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:34:56.672711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5ab99-94eb-40cd-9260-6b55ebdeaf57 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:56.674173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a2f5ab99-94eb-40cd-9260-6b55ebdeaf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:56.686157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5ab99-94eb-40cd-9260-6b55ebdeaf57 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:56.691205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:56.691996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:56.692282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:56.692910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] PATH_INFO: `/v3/projects/3fd2bc8a84e84fc08bd80d78f2c5eca9` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:56.694448 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd2bc8a84e84fc08bd80d78f2c5eca9)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:56.697938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:56.747562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd2bc8a84e84fc08bd80d78f2c5eca9', 'request_id': 'req-19e53fb4-39ca-449e-9fe4-6beb31346911'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.753792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd2bc8a84e84fc08bd80d78f2c5eca9', 'request_id': 'req-19e53fb4-39ca-449e-9fe4-6beb31346911'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.759616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd2bc8a84e84fc08bd80d78f2c5eca9', 'request_id': 'req-19e53fb4-39ca-449e-9fe4-6beb31346911'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.775373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] The token cache is being invalidate because project 3fd2bc8a84e84fc08bd80d78f2c5eca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:34:56.775800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.776482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.778057 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1123/2241] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:34:56 2022] DELETE /identity/v3/projects/3fd2bc8a84e84fc08bd80d78f2c5eca9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:56.782015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcb096-22b9-45b0-815c-9b94458d46c1 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:56.792678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcb096-22b9-45b0-815c-9b94458d46c1 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:56.801030 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:56.855730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf36896-045c-43c5-a5e3-0d5a70c30e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:56.856370 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf36896-045c-43c5-a5e3-0d5a70c30e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:56.856651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf36896-045c-43c5-a5e3-0d5a70c30e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:56.856959 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf36896-045c-43c5-a5e3-0d5a70c30e8c admin admin] PATH_INFO: `/v3/users/5d3b8fd1dbe54b0aa593c249d9f8a731` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:56.858096 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf36896-045c-43c5-a5e3-0d5a70c30e8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d3b8fd1dbe54b0aa593c249d9f8a731)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:56.860040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf36896-045c-43c5-a5e3-0d5a70c30e8c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:56.911370 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0bf36896-045c-43c5-a5e3-0d5a70c30e8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d3b8fd1dbe54b0aa593c249d9f8a731'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.912074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0bf36896-045c-43c5-a5e3-0d5a70c30e8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d3b8fd1dbe54b0aa593c249d9f8a731'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.912672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0bf36896-045c-43c5-a5e3-0d5a70c30e8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d3b8fd1dbe54b0aa593c249d9f8a731'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.926676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0bf36896-045c-43c5-a5e3-0d5a70c30e8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d3b8fd1dbe54b0aa593c249d9f8a731'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.934427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0bf36896-045c-43c5-a5e3-0d5a70c30e8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d3b8fd1dbe54b0aa593c249d9f8a731'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:56.947638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1119/2242] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:34:56 2022] DELETE /identity/v3/users/5d3b8fd1dbe54b0aa593c249d9f8a731 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:34:57.437422 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:57.438973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:57.451791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e53fb4-39ca-449e-9fe4-6beb31346911 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:57.497841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:57.515487 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abc48a-5673-480c-84c0-8f56f85151c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:57.516281 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abc48a-5673-480c-84c0-8f56f85151c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:57.516665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abc48a-5673-480c-84c0-8f56f85151c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:57.517071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abc48a-5673-480c-84c0-8f56f85151c8 admin admin] PATH_INFO: `/v3/projects/075a043bf9cb4c4cb854cd710db906ef` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:57.518312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abc48a-5673-480c-84c0-8f56f85151c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=075a043bf9cb4c4cb854cd710db906ef)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:57.520274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abc48a-5673-480c-84c0-8f56f85151c8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:57.574164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f6abc48a-5673-480c-84c0-8f56f85151c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '075a043bf9cb4c4cb854cd710db906ef', 'request_id': 'req-f6abc48a-5673-480c-84c0-8f56f85151c8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:57.581998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f6abc48a-5673-480c-84c0-8f56f85151c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '075a043bf9cb4c4cb854cd710db906ef', 'request_id': 'req-f6abc48a-5673-480c-84c0-8f56f85151c8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:57.588610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f6abc48a-5673-480c-84c0-8f56f85151c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '075a043bf9cb4c4cb854cd710db906ef', 'request_id': 'req-f6abc48a-5673-480c-84c0-8f56f85151c8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:57.604996 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f6abc48a-5673-480c-84c0-8f56f85151c8 admin admin] The token cache is being invalidate because project 075a043bf9cb4c4cb854cd710db906ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:34:57.605284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f6abc48a-5673-480c-84c0-8f56f85151c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:57.605916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f6abc48a-5673-480c-84c0-8f56f85151c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:34:57.607641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1124/2243] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:34:57 2022] DELETE /identity/v3/projects/075a043bf9cb4c4cb854cd710db906ef => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:57.612364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f69166-9ba6-44a6-8608-16fa55eb0800 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:57.612869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f69166-9ba6-44a6-8608-16fa55eb0800 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:57.613041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f69166-9ba6-44a6-8608-16fa55eb0800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:57.659317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-90f69166-9ba6-44a6-8608-16fa55eb0800 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:57.660193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-90f69166-9ba6-44a6-8608-16fa55eb0800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:57.734673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1120/2244] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:57.742995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7860695-54d0-4fab-b710-57f5c425ce47 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:57.743207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7860695-54d0-4fab-b710-57f5c425ce47 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:57.743482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7860695-54d0-4fab-b710-57f5c425ce47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:57.767175 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-c7860695-54d0-4fab-b710-57f5c425ce47 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:57.768463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c7860695-54d0-4fab-b710-57f5c425ce47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:57.779979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1125/2245] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:34:57.784230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f69166-9ba6-44a6-8608-16fa55eb0800 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:57.795652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f69166-9ba6-44a6-8608-16fa55eb0800 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:57.800536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c45dd-5303-4544-8727-17a13721e2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:57.801193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c45dd-5303-4544-8727-17a13721e2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:57.801478 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c45dd-5303-4544-8727-17a13721e2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:57.801756 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c45dd-5303-4544-8727-17a13721e2b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:57.803080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c45dd-5303-4544-8727-17a13721e2b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:34:57.803443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c45dd-5303-4544-8727-17a13721e2b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:57.805097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c45dd-5303-4544-8727-17a13721e2b2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:57.817108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1121/2246] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:34:57.821660 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7860695-54d0-4fab-b710-57f5c425ce47 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:57.831326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7860695-54d0-4fab-b710-57f5c425ce47 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:57.836241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c60e3d-dc08-41f1-bbd0-e60bf9f8c11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:57.836964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c60e3d-dc08-41f1-bbd0-e60bf9f8c11d admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:57.837309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c60e3d-dc08-41f1-bbd0-e60bf9f8c11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:57.837647 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c60e3d-dc08-41f1-bbd0-e60bf9f8c11d admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:57.838346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c60e3d-dc08-41f1-bbd0-e60bf9f8c11d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:57.840310 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c60e3d-dc08-41f1-bbd0-e60bf9f8c11d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:57.868987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1126/2247] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:34:57 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:57.873118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c45dd-5303-4544-8727-17a13721e2b2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:57.883731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c45dd-5303-4544-8727-17a13721e2b2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:57.897804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:34:57.945025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8301163-7f1d-46cf-b66d-0d7149782258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:57.945632 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8301163-7f1d-46cf-b66d-0d7149782258 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:57.945905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8301163-7f1d-46cf-b66d-0d7149782258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:57.946174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8301163-7f1d-46cf-b66d-0d7149782258 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:57.946804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8301163-7f1d-46cf-b66d-0d7149782258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:57.948720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8301163-7f1d-46cf-b66d-0d7149782258 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:57.970678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1122/2248] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:34:57 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:34:57.975550 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c60e3d-dc08-41f1-bbd0-e60bf9f8c11d admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:57.986315 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c60e3d-dc08-41f1-bbd0-e60bf9f8c11d admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:57.991326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1318774-ea34-49a4-8c01-ecb7fc49e79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:57.991967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1318774-ea34-49a4-8c01-ecb7fc49e79f admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:57.992243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1318774-ea34-49a4-8c01-ecb7fc49e79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:57.992513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1318774-ea34-49a4-8c01-ecb7fc49e79f admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:57.993282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1318774-ea34-49a4-8c01-ecb7fc49e79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:57.994847 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1318774-ea34-49a4-8c01-ecb7fc49e79f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:58.005277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1127/2249] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:58.009336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8301163-7f1d-46cf-b66d-0d7149782258 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:58.019842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8301163-7f1d-46cf-b66d-0d7149782258 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:58.025249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd5b95-0a50-4a9b-8136-509d844c1d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:58.027027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd5b95-0a50-4a9b-8136-509d844c1d3b admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:58.027027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd5b95-0a50-4a9b-8136-509d844c1d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:58.027027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd5b95-0a50-4a9b-8136-509d844c1d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:58.027725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd5b95-0a50-4a9b-8136-509d844c1d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:58.028680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd5b95-0a50-4a9b-8136-509d844c1d3b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:58.040711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1123/2250] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:34:58.044815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1318774-ea34-49a4-8c01-ecb7fc49e79f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:58.056099 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1318774-ea34-49a4-8c01-ecb7fc49e79f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:58.064469 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a1c6d-b754-4008-8c35-92cc1bddda21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:58.070944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a1c6d-b754-4008-8c35-92cc1bddda21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:58.070944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a1c6d-b754-4008-8c35-92cc1bddda21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:58.071121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a1c6d-b754-4008-8c35-92cc1bddda21 admin admin] PATH_INFO: `/v3/projects/a41afd08d1324396940e3acaf49a34b3/users/cf1371337c03435faa5f1db60168cfbe/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:58.090646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a1c6d-b754-4008-8c35-92cc1bddda21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a41afd08d1324396940e3acaf49a34b3, user_id=cf1371337c03435faa5f1db60168cfbe, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:58.093399 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a1c6d-b754-4008-8c35-92cc1bddda21 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:58.116010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1128/2251] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:34:58 2022] PUT /identity/v3/projects/a41afd08d1324396940e3acaf49a34b3/users/cf1371337c03435faa5f1db60168cfbe/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:34:58.121673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae511a-63f6-4020-b50f-1ce6afef82d2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:58.122777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae511a-63f6-4020-b50f-1ce6afef82d2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:58.123243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae511a-63f6-4020-b50f-1ce6afef82d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:58.159222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8bae511a-63f6-4020-b50f-1ce6afef82d2 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:34:58.162667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8bae511a-63f6-4020-b50f-1ce6afef82d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:58.235923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1124/2252] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:34:58.246636 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a1c6d-b754-4008-8c35-92cc1bddda21 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:34:58.248438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ce6a1c6d-b754-4008-8c35-92cc1bddda21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:34:58.307955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a1c6d-b754-4008-8c35-92cc1bddda21 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:34:58.311700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7970116-dd61-4f6f-8dad-ef4265bb59db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:34:58.312306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7970116-dd61-4f6f-8dad-ef4265bb59db service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:34:58.312583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7970116-dd61-4f6f-8dad-ef4265bb59db service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:34:58.312886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7970116-dd61-4f6f-8dad-ef4265bb59db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:34:58.322940 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7970116-dd61-4f6f-8dad-ef4265bb59db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:34:58.324400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7970116-dd61-4f6f-8dad-ef4265bb59db service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:34:58.335586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1129/2253] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:35:03.712126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d8f60-08ee-4b2d-a1a3-a19f27844732 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:03.712643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d8f60-08ee-4b2d-a1a3-a19f27844732 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:03.712869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d8f60-08ee-4b2d-a1a3-a19f27844732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:03.745216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-523d8f60-08ee-4b2d-a1a3-a19f27844732 None None] MFA Rules not processed for user `cf1371337c03435faa5f1db60168cfbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:35:03.747429 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-523d8f60-08ee-4b2d-a1a3-a19f27844732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:03.832802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1125/2254] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:35:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:35:03.834604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7970116-dd61-4f6f-8dad-ef4265bb59db service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:03.844583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7970116-dd61-4f6f-8dad-ef4265bb59db service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:03.850840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780621d-00a9-4a17-9e38-5718504a8f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:03.851562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780621d-00a9-4a17-9e38-5718504a8f15 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:03.851914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780621d-00a9-4a17-9e38-5718504a8f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:03.852257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780621d-00a9-4a17-9e38-5718504a8f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:03.852994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780621d-00a9-4a17-9e38-5718504a8f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:03.855005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780621d-00a9-4a17-9e38-5718504a8f15 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:03.869448 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:35:03.890969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1130/2255] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:35:03 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:35:03.894791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d8f60-08ee-4b2d-a1a3-a19f27844732 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:03.906689 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d8f60-08ee-4b2d-a1a3-a19f27844732 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:03.938630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:35:03.979089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fd725-f03b-4279-a049-69fd11e8d4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:03.979709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fd725-f03b-4279-a049-69fd11e8d4c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:03.979992 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fd725-f03b-4279-a049-69fd11e8d4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:03.980268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fd725-f03b-4279-a049-69fd11e8d4c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:03.981217 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94fd725-f03b-4279-a049-69fd11e8d4c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:03.983462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94fd725-f03b-4279-a049-69fd11e8d4c1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:04.007872 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1126/2256] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:35:03 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:35:04.012427 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780621d-00a9-4a17-9e38-5718504a8f15 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:04.024706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780621d-00a9-4a17-9e38-5718504a8f15 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:04.030034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690ed00-2b63-4b8e-93ab-37fdb13a1a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:04.030720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690ed00-2b63-4b8e-93ab-37fdb13a1a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:04.031064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690ed00-2b63-4b8e-93ab-37fdb13a1a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:04.031410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690ed00-2b63-4b8e-93ab-37fdb13a1a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:04.032183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690ed00-2b63-4b8e-93ab-37fdb13a1a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:04.034142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690ed00-2b63-4b8e-93ab-37fdb13a1a9a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:04.046422 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1131/2257] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:35:04.050545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fd725-f03b-4279-a049-69fd11e8d4c1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:04.061783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fd725-f03b-4279-a049-69fd11e8d4c1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:04.067127 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75727db-41e6-4631-93f3-89f5a5e218b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:04.067932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75727db-41e6-4631-93f3-89f5a5e218b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:04.068229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75727db-41e6-4631-93f3-89f5a5e218b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:04.068506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75727db-41e6-4631-93f3-89f5a5e218b6 admin admin] PATH_INFO: `/v3/projects/d60b3e22e78149a1bb5eab7b3805f48c/users/d62cc27342c54b4fb961db960577e748/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:04.089873 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75727db-41e6-4631-93f3-89f5a5e218b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d60b3e22e78149a1bb5eab7b3805f48c, user_id=d62cc27342c54b4fb961db960577e748, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:04.092727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75727db-41e6-4631-93f3-89f5a5e218b6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:04.108401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1127/2258] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:35:04 2022] PUT /identity/v3/projects/d60b3e22e78149a1bb5eab7b3805f48c/users/d62cc27342c54b4fb961db960577e748/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:35:04.112706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690ed00-2b63-4b8e-93ab-37fdb13a1a9a admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:04.122827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690ed00-2b63-4b8e-93ab-37fdb13a1a9a admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:04.181236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68300704-0215-437a-8fea-a3b7a9a23e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:04.181843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68300704-0215-437a-8fea-a3b7a9a23e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:04.182121 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68300704-0215-437a-8fea-a3b7a9a23e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:04.182393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68300704-0215-437a-8fea-a3b7a9a23e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:04.183026 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68300704-0215-437a-8fea-a3b7a9a23e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:04.184611 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68300704-0215-437a-8fea-a3b7a9a23e37 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:04.194953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1132/2259] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:35:04.199080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75727db-41e6-4631-93f3-89f5a5e218b6 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:04.210712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75727db-41e6-4631-93f3-89f5a5e218b6 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:04.215440 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bba1c-0b9f-4783-aeb9-8aabe5cc8005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:04.216117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bba1c-0b9f-4783-aeb9-8aabe5cc8005 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:04.216392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bba1c-0b9f-4783-aeb9-8aabe5cc8005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:04.216662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bba1c-0b9f-4783-aeb9-8aabe5cc8005 admin admin] PATH_INFO: `/v3/projects/d60b3e22e78149a1bb5eab7b3805f48c/users/d62cc27342c54b4fb961db960577e748/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:04.218640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bba1c-0b9f-4783-aeb9-8aabe5cc8005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d60b3e22e78149a1bb5eab7b3805f48c, user_id=d62cc27342c54b4fb961db960577e748, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:04.221307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bba1c-0b9f-4783-aeb9-8aabe5cc8005 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:04.233370 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1128/2260] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:35:04 2022] PUT /identity/v3/projects/d60b3e22e78149a1bb5eab7b3805f48c/users/d62cc27342c54b4fb961db960577e748/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:35:05.172120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68300704-0215-437a-8fea-a3b7a9a23e37 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:05.175369 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-68300704-0215-437a-8fea-a3b7a9a23e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:05.311744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68300704-0215-437a-8fea-a3b7a9a23e37 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:05.402839 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a5e95-2bbf-4a70-8778-a07e822b9af5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:05.403687 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a5e95-2bbf-4a70-8778-a07e822b9af5 service placement] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:05.404287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a5e95-2bbf-4a70-8778-a07e822b9af5 service placement] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:05.406230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a5e95-2bbf-4a70-8778-a07e822b9af5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:05.410801 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9b4a5e95-2bbf-4a70-8778-a07e822b9af5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:05.439543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a5e95-2bbf-4a70-8778-a07e822b9af5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:05.445965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a5e95-2bbf-4a70-8778-a07e822b9af5 service placement] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:05.471432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1133/2261] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:35:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:35:08.061437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd0be9-0e43-4ba7-a249-4355b9b0ddae None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:08.061977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd0be9-0e43-4ba7-a249-4355b9b0ddae None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:08.061977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd0be9-0e43-4ba7-a249-4355b9b0ddae None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:08.063591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1129/2262] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:35:08.075610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668dcbd-89b6-490b-b6b4-044b1ece0f9e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:08.076191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668dcbd-89b6-490b-b6b4-044b1ece0f9e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:08.076191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668dcbd-89b6-490b-b6b4-044b1ece0f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:08.106056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b668dcbd-89b6-490b-b6b4-044b1ece0f9e None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:35:08.110029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b668dcbd-89b6-490b-b6b4-044b1ece0f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:08.118966 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:35:08.232669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1134/2263] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:35:08.248813 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd0be9-0e43-4ba7-a249-4355b9b0ddae None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:08.251903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c5cd0be9-0e43-4ba7-a249-4355b9b0ddae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:08.370817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd0be9-0e43-4ba7-a249-4355b9b0ddae None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:08.375577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4f06b-4aff-459a-b715-448b673645c6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '748ad823b944441d907994cda6d8ac7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:08.376342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4f06b-4aff-459a-b715-448b673645c6 service glance] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:08.376634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4f06b-4aff-459a-b715-448b673645c6 service glance] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:08.376919 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4f06b-4aff-459a-b715-448b673645c6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:08.389190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4f06b-4aff-459a-b715-448b673645c6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:08.390897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4f06b-4aff-459a-b715-448b673645c6 service glance] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:08.404062 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1130/2264] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:35:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:35:08.490165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d947dc-d73f-4c69-b9e6-1c76bbd48049 None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:08.490454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d947dc-d73f-4c69-b9e6-1c76bbd48049 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:08.490733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d947dc-d73f-4c69-b9e6-1c76bbd48049 None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:08.492491 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1135/2265] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:35:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:35:08.497852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc6fcb-074c-4269-970c-7b9d346a37a9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:08.498141 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc6fcb-074c-4269-970c-7b9d346a37a9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:08.498415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc6fcb-074c-4269-970c-7b9d346a37a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:08.523999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f4bc6fcb-074c-4269-970c-7b9d346a37a9 None None] MFA Rules not processed for user `9e0b7efc488a42ffab8ba7e2d101ced4`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:35:08.524982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f4bc6fcb-074c-4269-970c-7b9d346a37a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:08.603293 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1131/2266] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:35:08.609736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d947dc-d73f-4c69-b9e6-1c76bbd48049 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:08.611514 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c2d947dc-d73f-4c69-b9e6-1c76bbd48049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:08.626713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d947dc-d73f-4c69-b9e6-1c76bbd48049 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:08.631195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f59ee1-f7e0-4890-990b-255f2bf9d196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:08.632027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59ee1-f7e0-4890-990b-255f2bf9d196 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:08.632259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59ee1-f7e0-4890-990b-255f2bf9d196 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:08.632548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59ee1-f7e0-4890-990b-255f2bf9d196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:08.641788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f59ee1-f7e0-4890-990b-255f2bf9d196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:08.643472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f59ee1-f7e0-4890-990b-255f2bf9d196 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:08.656779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1136/2267] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:35:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:35:08.791256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290d76c-b5df-45ba-b5ee-ac3bc9b64b73 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:08.791879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290d76c-b5df-45ba-b5ee-ac3bc9b64b73 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:08.792279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290d76c-b5df-45ba-b5ee-ac3bc9b64b73 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:08.793952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1132/2268] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:35:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:35:08.803666 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b0085-7504-438d-9704-86c6a88232a6 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:08.803968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b0085-7504-438d-9704-86c6a88232a6 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:08.805141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b0085-7504-438d-9704-86c6a88232a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:08.835957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-5f6b0085-7504-438d-9704-86c6a88232a6 None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:35:08.837360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5f6b0085-7504-438d-9704-86c6a88232a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:08.857608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1137/2269] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:35:08.929803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290d76c-b5df-45ba-b5ee-ac3bc9b64b73 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:08.948037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290d76c-b5df-45ba-b5ee-ac3bc9b64b73 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:08.953799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4943f83-f687-46cc-9bbf-71dee4836bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:08.956313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4943f83-f687-46cc-9bbf-71dee4836bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:08.957194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4943f83-f687-46cc-9bbf-71dee4836bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:08.965786 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4943f83-f687-46cc-9bbf-71dee4836bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:08.993846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4943f83-f687-46cc-9bbf-71dee4836bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:08.995282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4943f83-f687-46cc-9bbf-71dee4836bc6 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:09.000519 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:35:09.016215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1133/2270] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:35:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:35:09.850450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4bc2e-229f-4f92-aeea-c3ae565f6f48 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:09.850988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4bc2e-229f-4f92-aeea-c3ae565f6f48 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:09.851082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4bc2e-229f-4f92-aeea-c3ae565f6f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:09.878557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-cfc4bc2e-229f-4f92-aeea-c3ae565f6f48 None None] MFA Rules not processed for user `d62cc27342c54b4fb961db960577e748`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:35:09.880586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-cfc4bc2e-229f-4f92-aeea-c3ae565f6f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:09.953171 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1138/2271] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:35:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:35:09.967150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4943f83-f687-46cc-9bbf-71dee4836bc6 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:09.978014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4943f83-f687-46cc-9bbf-71dee4836bc6 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:09.981659 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25c635-6159-49a3-9f2e-600b381d53e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '748ad823b944441d907994cda6d8ac7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:09.982672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25c635-6159-49a3-9f2e-600b381d53e1 service glance] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:09.983015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25c635-6159-49a3-9f2e-600b381d53e1 service glance] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:09.983369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25c635-6159-49a3-9f2e-600b381d53e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:10.048125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25c635-6159-49a3-9f2e-600b381d53e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:10.049660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25c635-6159-49a3-9f2e-600b381d53e1 service glance] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:10.070240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1134/2272] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:35:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:35:10.243942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb20e2b-3dd6-4363-ad4b-e30f10839b35 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:10.244235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb20e2b-3dd6-4363-ad4b-e30f10839b35 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:10.244625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb20e2b-3dd6-4363-ad4b-e30f10839b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:10.265245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-dbb20e2b-3dd6-4363-ad4b-e30f10839b35 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:35:10.266747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-dbb20e2b-3dd6-4363-ad4b-e30f10839b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:10.328104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1139/2273] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:35:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:35:10.352824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25c635-6159-49a3-9f2e-600b381d53e1 service glance] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:10.355163 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9a25c635-6159-49a3-9f2e-600b381d53e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:10.486319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25c635-6159-49a3-9f2e-600b381d53e1 service glance] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:10.497030 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76ca5d-3b12-4ee8-935b-cd7300cc5588 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:10.497700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76ca5d-3b12-4ee8-935b-cd7300cc5588 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:10.497982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76ca5d-3b12-4ee8-935b-cd7300cc5588 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:10.498259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76ca5d-3b12-4ee8-935b-cd7300cc5588 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:10.511527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76ca5d-3b12-4ee8-935b-cd7300cc5588 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:10.513223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76ca5d-3b12-4ee8-935b-cd7300cc5588 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:10.522876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1135/2274] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:35:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:35:14.739506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba91534-598b-483d-9f0c-e6fad1949005 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:14.741045 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba91534-598b-483d-9f0c-e6fad1949005 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:14.741540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba91534-598b-483d-9f0c-e6fad1949005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:14.770436 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-fba91534-598b-483d-9f0c-e6fad1949005 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:35:14.772424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-fba91534-598b-483d-9f0c-e6fad1949005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:14.780593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1140/2275] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:35:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:35:14.793204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76ca5d-3b12-4ee8-935b-cd7300cc5588 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:14.813118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76ca5d-3b12-4ee8-935b-cd7300cc5588 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:14.825294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f196c-9046-42f0-962b-e31bf26b07ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:14.826297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f196c-9046-42f0-962b-e31bf26b07ee service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:14.826687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f196c-9046-42f0-962b-e31bf26b07ee service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:14.827186 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f196c-9046-42f0-962b-e31bf26b07ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:14.846881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f196c-9046-42f0-962b-e31bf26b07ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:14.852347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f196c-9046-42f0-962b-e31bf26b07ee service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:14.869010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1136/2276] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:35:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:35:20.979680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16114af7-2d88-4da2-8528-dd04316855b0 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:20.980284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16114af7-2d88-4da2-8528-dd04316855b0 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:20.980284 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16114af7-2d88-4da2-8528-dd04316855b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:20.990205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:35:21.003316 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-16114af7-2d88-4da2-8528-dd04316855b0 None None] MFA Rules not processed for user `cf1371337c03435faa5f1db60168cfbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:35:21.004600 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-16114af7-2d88-4da2-8528-dd04316855b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:21.012159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1141/2277] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:35:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:35:24.514999 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3aee2e-0121-413e-a13b-4676a538e920 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:24.515577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3aee2e-0121-413e-a13b-4676a538e920 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:24.515577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3aee2e-0121-413e-a13b-4676a538e920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:24.537153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0e3aee2e-0121-413e-a13b-4676a538e920 None None] MFA Rules not processed for user `d9e505bb350e4134976000a83bdd4fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:35:24.538385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0e3aee2e-0121-413e-a13b-4676a538e920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:24.597607 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1137/2278] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:35:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:35:30.999546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:35:31.981333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ea71-b2b1-4aa3-b617-1bdc8c73ed46 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:31.981537 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ea71-b2b1-4aa3-b617-1bdc8c73ed46 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:31.981838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ea71-b2b1-4aa3-b617-1bdc8c73ed46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:32.010343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-44d8ea71-b2b1-4aa3-b617-1bdc8c73ed46 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:35:32.011345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-44d8ea71-b2b1-4aa3-b617-1bdc8c73ed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:32.019954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1142/2279] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:35:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:35:32.043035 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3aee2e-0121-413e-a13b-4676a538e920 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:32.060294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3aee2e-0121-413e-a13b-4676a538e920 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:32.066683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd681a-a186-411c-8915-8f3572e7cb28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:32.067565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd681a-a186-411c-8915-8f3572e7cb28 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:32.068337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd681a-a186-411c-8915-8f3572e7cb28 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:32.068802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd681a-a186-411c-8915-8f3572e7cb28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:32.083538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd681a-a186-411c-8915-8f3572e7cb28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:32.085633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd681a-a186-411c-8915-8f3572e7cb28 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:32.097972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1138/2280] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:35:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:35:42.003152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:35:42.093415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:35:53.014628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f9fe0-e619-4f3b-b5c1-742a5fb40b3b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:53.015089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f9fe0-e619-4f3b-b5c1-742a5fb40b3b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:53.015186 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f9fe0-e619-4f3b-b5c1-742a5fb40b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:53.035546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d56f9fe0-e619-4f3b-b5c1-742a5fb40b3b None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:35:53.036799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d56f9fe0-e619-4f3b-b5c1-742a5fb40b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:35:53.043817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1143/2281] 10.222.0.55 () {58 vars in 967 bytes} [Fri Aug 19 16:35:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:35:53.057385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd681a-a186-411c-8915-8f3572e7cb28 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:35:53.069616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd681a-a186-411c-8915-8f3572e7cb28 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:35:53.075150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1595f11-1e73-41dc-9bb3-8edf40defc14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:35:53.075765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595f11-1e73-41dc-9bb3-8edf40defc14 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:35:53.076043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595f11-1e73-41dc-9bb3-8edf40defc14 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:35:53.076315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595f11-1e73-41dc-9bb3-8edf40defc14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:35:53.086039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1595f11-1e73-41dc-9bb3-8edf40defc14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:35:53.087624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1595f11-1e73-41dc-9bb3-8edf40defc14 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:35:53.096231 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1139/2282] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:35:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:36:03.030065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:36:03.093150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=6 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:36:22.393409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f9fe0-e619-4f3b-b5c1-742a5fb40b3b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:22.402699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f9fe0-e619-4f3b-b5c1-742a5fb40b3b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:22.406249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b4028-4085-4308-be66-26f1a31d2a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:22.406842 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4028-4085-4308-be66-26f1a31d2a2b service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:22.407122 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4028-4085-4308-be66-26f1a31d2a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:22.407392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4028-4085-4308-be66-26f1a31d2a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:22.416819 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b4028-4085-4308-be66-26f1a31d2a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:22.418236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b4028-4085-4308-be66-26f1a31d2a2b service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:22.428282 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1144/2283] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:36:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:36:32.423364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=6 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:36:33.913881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93da64-6594-4f72-ae56-032fb6b85d67 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:33.914430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93da64-6594-4f72-ae56-032fb6b85d67 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:33.914573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93da64-6594-4f72-ae56-032fb6b85d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:33.937643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1a93da64-6594-4f72-ae56-032fb6b85d67 None None] MFA Rules not processed for user `7936bf23a031418b8ce304aa3bbcf8a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:36:33.939178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1a93da64-6594-4f72-ae56-032fb6b85d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:36:34.004952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1140/2284] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:36:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:36:34.118383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a57df2-4351-4fe2-b46f-51f43471eb55 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:34.118383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a57df2-4351-4fe2-b46f-51f43471eb55 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:34.120469 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a57df2-4351-4fe2-b46f-51f43471eb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:34.140135 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-66a57df2-4351-4fe2-b46f-51f43471eb55 None None] MFA Rules not processed for user `9b0455ae3ec94c929c940df6a0f6687d`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:36:34.141372 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-66a57df2-4351-4fe2-b46f-51f43471eb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:36:34.197910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1145/2285] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:36:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:36:43.996974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=25 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:36:44.190032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=25 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:36:47.882702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93da64-6594-4f72-ae56-032fb6b85d67 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:47.895216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93da64-6594-4f72-ae56-032fb6b85d67 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:47.899198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816e186-ded5-41c5-8c6d-505275a34f50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:47.899832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816e186-ded5-41c5-8c6d-505275a34f50 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:47.900117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816e186-ded5-41c5-8c6d-505275a34f50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:47.900423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816e186-ded5-41c5-8c6d-505275a34f50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:47.903064 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2816e186-ded5-41c5-8c6d-505275a34f50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:36:47.920652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816e186-ded5-41c5-8c6d-505275a34f50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:47.922159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816e186-ded5-41c5-8c6d-505275a34f50 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:47.935939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1141/2286] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:36:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:36:50.407782 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a57df2-4351-4fe2-b46f-51f43471eb55 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:50.417394 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a57df2-4351-4fe2-b46f-51f43471eb55 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:50.420978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ab32f-31ea-4ed9-bb6a-1c9964cd86d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:50.421573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ab32f-31ea-4ed9-bb6a-1c9964cd86d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:50.421852 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ab32f-31ea-4ed9-bb6a-1c9964cd86d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:50.422124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ab32f-31ea-4ed9-bb6a-1c9964cd86d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:50.424331 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d84ab32f-31ea-4ed9-bb6a-1c9964cd86d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:36:50.437674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ab32f-31ea-4ed9-bb6a-1c9964cd86d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:50.439094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ab32f-31ea-4ed9-bb6a-1c9964cd86d4 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:50.450851 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1146/2287] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:36:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:36:56.749584 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816e186-ded5-41c5-8c6d-505275a34f50 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:56.751107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2816e186-ded5-41c5-8c6d-505275a34f50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:36:56.765693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816e186-ded5-41c5-8c6d-505275a34f50 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:56.772501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d238b5-696e-4871-9aa5-c8cadb35f04d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:56.773196 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d238b5-696e-4871-9aa5-c8cadb35f04d service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:56.773721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d238b5-696e-4871-9aa5-c8cadb35f04d service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:56.774000 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d238b5-696e-4871-9aa5-c8cadb35f04d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:56.775738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-02d238b5-696e-4871-9aa5-c8cadb35f04d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:36:56.840773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d238b5-696e-4871-9aa5-c8cadb35f04d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:56.842388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d238b5-696e-4871-9aa5-c8cadb35f04d service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:56.854801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1142/2288] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:36:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:36:58.000088 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ab32f-31ea-4ed9-bb6a-1c9964cd86d4 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:58.001592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d84ab32f-31ea-4ed9-bb6a-1c9964cd86d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:36:58.066363 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ab32f-31ea-4ed9-bb6a-1c9964cd86d4 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:58.070950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdddd5d-98d6-4437-a1fc-5ceb2e32c4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:58.071568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdddd5d-98d6-4437-a1fc-5ceb2e32c4d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:58.071849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdddd5d-98d6-4437-a1fc-5ceb2e32c4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:58.072118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdddd5d-98d6-4437-a1fc-5ceb2e32c4d5 admin admin] PATH_INFO: `/v3/users/d9e505bb350e4134976000a83bdd4fe1` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:58.073264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdddd5d-98d6-4437-a1fc-5ceb2e32c4d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9e505bb350e4134976000a83bdd4fe1)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:58.075411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdddd5d-98d6-4437-a1fc-5ceb2e32c4d5 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:58.119346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-bfdddd5d-98d6-4437-a1fc-5ceb2e32c4d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9e505bb350e4134976000a83bdd4fe1'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:36:58.119911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-bfdddd5d-98d6-4437-a1fc-5ceb2e32c4d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9e505bb350e4134976000a83bdd4fe1'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:36:58.120498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-bfdddd5d-98d6-4437-a1fc-5ceb2e32c4d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9e505bb350e4134976000a83bdd4fe1'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:36:58.127936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-bfdddd5d-98d6-4437-a1fc-5ceb2e32c4d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9e505bb350e4134976000a83bdd4fe1'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:36:58.139523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-bfdddd5d-98d6-4437-a1fc-5ceb2e32c4d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9e505bb350e4134976000a83bdd4fe1'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:36:58.152603 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1147/2289] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:36:57 2022] DELETE /identity/v3/users/d9e505bb350e4134976000a83bdd4fe1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:36:58.326851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d238b5-696e-4871-9aa5-c8cadb35f04d service nova] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:58.338842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d238b5-696e-4871-9aa5-c8cadb35f04d service nova] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:58.414194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:58.414861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:58.415144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:58.415417 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643 admin admin] PATH_INFO: `/v3/projects/b34c07333ddf4a19b35424d47d19dc60` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:58.416529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34c07333ddf4a19b35424d47d19dc60)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:58.418485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:58.467592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34c07333ddf4a19b35424d47d19dc60', 'request_id': 'req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:36:58.474134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34c07333ddf4a19b35424d47d19dc60', 'request_id': 'req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:36:58.480861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34c07333ddf4a19b35424d47d19dc60', 'request_id': 'req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:36:58.494670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643 admin admin] The token cache is being invalidate because project b34c07333ddf4a19b35424d47d19dc60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:36:58.494965 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:36:58.495592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b1f6e77d-07e6-4b71-a4e1-83bba76ab643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:36:58.497124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1143/2290] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:36:58 2022] DELETE /identity/v3/projects/b34c07333ddf4a19b35424d47d19dc60 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:36:58.502103 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3556e0-420c-42c3-9baf-4b213bf78d64 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:58.502501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3556e0-420c-42c3-9baf-4b213bf78d64 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:58.502726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3556e0-420c-42c3-9baf-4b213bf78d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:58.522135 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-6f3556e0-420c-42c3-9baf-4b213bf78d64 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:36:58.523341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6f3556e0-420c-42c3-9baf-4b213bf78d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:36:58.583827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1148/2291] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:36:58.592683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4e954-fc73-49a1-839d-6e898fcd85f9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:58.593014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4e954-fc73-49a1-839d-6e898fcd85f9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:58.593298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4e954-fc73-49a1-839d-6e898fcd85f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:58.613548 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-34a4e954-fc73-49a1-839d-6e898fcd85f9 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:36:58.614780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-34a4e954-fc73-49a1-839d-6e898fcd85f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:36:58.622973 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1144/2292] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:36:58.627460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3556e0-420c-42c3-9baf-4b213bf78d64 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:58.637849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3556e0-420c-42c3-9baf-4b213bf78d64 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:58.643283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af292d5d-fe97-48ff-813e-a772d00e8b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:58.644313 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af292d5d-fe97-48ff-813e-a772d00e8b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:58.644880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af292d5d-fe97-48ff-813e-a772d00e8b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:58.645278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af292d5d-fe97-48ff-813e-a772d00e8b68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:58.646035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af292d5d-fe97-48ff-813e-a772d00e8b68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:36:58.646404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af292d5d-fe97-48ff-813e-a772d00e8b68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:58.648077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af292d5d-fe97-48ff-813e-a772d00e8b68 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:58.649874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:36:58.668495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1149/2293] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:36:58.669329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4e954-fc73-49a1-839d-6e898fcd85f9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:58.678875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4e954-fc73-49a1-839d-6e898fcd85f9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:58.683516 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bde89-ccff-4b76-8307-e3d02be01766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:58.684167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bde89-ccff-4b76-8307-e3d02be01766 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:58.684443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bde89-ccff-4b76-8307-e3d02be01766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:58.684717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bde89-ccff-4b76-8307-e3d02be01766 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:58.685359 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bde89-ccff-4b76-8307-e3d02be01766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:58.687251 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bde89-ccff-4b76-8307-e3d02be01766 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:58.693356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:36:58.722997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1145/2294] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:36:58 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:36:58.724187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af292d5d-fe97-48ff-813e-a772d00e8b68 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:58.734139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af292d5d-fe97-48ff-813e-a772d00e8b68 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:58.793467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a8e7f-9413-4d73-89a6-8ccbbc6e4da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:58.794084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a8e7f-9413-4d73-89a6-8ccbbc6e4da2 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:58.794366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a8e7f-9413-4d73-89a6-8ccbbc6e4da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:58.794700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a8e7f-9413-4d73-89a6-8ccbbc6e4da2 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:58.795335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a8e7f-9413-4d73-89a6-8ccbbc6e4da2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:58.797385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a8e7f-9413-4d73-89a6-8ccbbc6e4da2 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:58.818516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1150/2295] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:36:58 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:36:58.822524 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bde89-ccff-4b76-8307-e3d02be01766 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:58.833670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bde89-ccff-4b76-8307-e3d02be01766 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:58.838247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea5958-e916-4909-91a4-b37c37265fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:58.838836 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea5958-e916-4909-91a4-b37c37265fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:58.839113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea5958-e916-4909-91a4-b37c37265fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:58.839382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea5958-e916-4909-91a4-b37c37265fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:58.840014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea5958-e916-4909-91a4-b37c37265fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:58.841619 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea5958-e916-4909-91a4-b37c37265fe8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:58.852128 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1146/2296] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:36:58.856054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a8e7f-9413-4d73-89a6-8ccbbc6e4da2 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:58.866025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a8e7f-9413-4d73-89a6-8ccbbc6e4da2 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:58.870882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e1101-fe8b-4688-9a55-328fc8b1934b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:58.871472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e1101-fe8b-4688-9a55-328fc8b1934b admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:58.871750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e1101-fe8b-4688-9a55-328fc8b1934b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:58.872018 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e1101-fe8b-4688-9a55-328fc8b1934b admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:58.872646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e1101-fe8b-4688-9a55-328fc8b1934b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:58.874256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e1101-fe8b-4688-9a55-328fc8b1934b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:58.884948 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1151/2297] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:36:58.889158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea5958-e916-4909-91a4-b37c37265fe8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:58.899500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea5958-e916-4909-91a4-b37c37265fe8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:58.904070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795b903-b18d-4027-a906-f9d1881bc330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:58.904705 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795b903-b18d-4027-a906-f9d1881bc330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:58.905019 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795b903-b18d-4027-a906-f9d1881bc330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:58.905289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795b903-b18d-4027-a906-f9d1881bc330 admin admin] PATH_INFO: `/v3/projects/5181e8549b3a439c8921319d6957f670/users/f989b6fded53458b85b39789a32a39e4/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:58.920362 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795b903-b18d-4027-a906-f9d1881bc330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5181e8549b3a439c8921319d6957f670, user_id=f989b6fded53458b85b39789a32a39e4, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:58.922944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795b903-b18d-4027-a906-f9d1881bc330 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:58.936066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1147/2298] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:36:58 2022] PUT /identity/v3/projects/5181e8549b3a439c8921319d6957f670/users/f989b6fded53458b85b39789a32a39e4/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:36:58.941713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785e2dc-70c2-4cb9-a66b-b73582b652b6 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:58.941998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785e2dc-70c2-4cb9-a66b-b73582b652b6 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:58.942269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785e2dc-70c2-4cb9-a66b-b73582b652b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:58.961787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3785e2dc-70c2-4cb9-a66b-b73582b652b6 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:36:58.963011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3785e2dc-70c2-4cb9-a66b-b73582b652b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:36:59.028722 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1152/2299] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:36:59.038338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795b903-b18d-4027-a906-f9d1881bc330 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:36:59.040014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6795b903-b18d-4027-a906-f9d1881bc330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:36:59.096783 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795b903-b18d-4027-a906-f9d1881bc330 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:36:59.100479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5995fd-7625-4aab-990f-a7c3db76c921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:36:59.101107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5995fd-7625-4aab-990f-a7c3db76c921 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:36:59.101390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5995fd-7625-4aab-990f-a7c3db76c921 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:36:59.101667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5995fd-7625-4aab-990f-a7c3db76c921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:36:59.110833 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5995fd-7625-4aab-990f-a7c3db76c921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:36:59.112298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5995fd-7625-4aab-990f-a7c3db76c921 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:36:59.121995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1148/2300] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:37:04.630450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cbf00-3935-4f42-8a20-630258c1147c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:04.630975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cbf00-3935-4f42-8a20-630258c1147c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:04.631123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cbf00-3935-4f42-8a20-630258c1147c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:04.665113 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d28cbf00-3935-4f42-8a20-630258c1147c None None] MFA Rules not processed for user `f989b6fded53458b85b39789a32a39e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:37:04.666686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d28cbf00-3935-4f42-8a20-630258c1147c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:04.735064 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f1c3f-e7bb-476f-bb5a-fc0573863df1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:04.735535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f1c3f-e7bb-476f-bb5a-fc0573863df1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:04.735748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f1c3f-e7bb-476f-bb5a-fc0573863df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:04.757296 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-324f1c3f-e7bb-476f-bb5a-fc0573863df1 None None] MFA Rules not processed for user `e09fd34737af41dc972d806dc3b0705e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:37:04.758626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-324f1c3f-e7bb-476f-bb5a-fc0573863df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:04.765853 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1153/2301] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:37:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:37:04.766778 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cbf00-3935-4f42-8a20-630258c1147c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:04.772744 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:37:04.781472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cbf00-3935-4f42-8a20-630258c1147c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:04.788208 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fdf2a-8659-48d7-9d67-f8e046bd3b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:04.789340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fdf2a-8659-48d7-9d67-f8e046bd3b0a admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:04.789696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fdf2a-8659-48d7-9d67-f8e046bd3b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:04.790066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fdf2a-8659-48d7-9d67-f8e046bd3b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:04.790885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fdf2a-8659-48d7-9d67-f8e046bd3b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:04.792934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fdf2a-8659-48d7-9d67-f8e046bd3b0a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:04.823622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1149/2302] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:37:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:37:04.823983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1154/2303] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:37:04 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:37:04.832264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f1c3f-e7bb-476f-bb5a-fc0573863df1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:04.843677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f1c3f-e7bb-476f-bb5a-fc0573863df1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:04.926950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e0432-d505-4e52-962d-96cd46882af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:04.927681 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e0432-d505-4e52-962d-96cd46882af4 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:04.928063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e0432-d505-4e52-962d-96cd46882af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:04.928489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e0432-d505-4e52-962d-96cd46882af4 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:04.937560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e0432-d505-4e52-962d-96cd46882af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:04.940450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e0432-d505-4e52-962d-96cd46882af4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:04.953537 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:37:04.969553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1150/2304] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:37:04 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:37:04.973587 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fdf2a-8659-48d7-9d67-f8e046bd3b0a admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:04.988815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fdf2a-8659-48d7-9d67-f8e046bd3b0a admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:04.997015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48506f5b-0da1-4195-8038-166a520a4b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:04.998493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48506f5b-0da1-4195-8038-166a520a4b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:04.998939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48506f5b-0da1-4195-8038-166a520a4b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:04.999573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48506f5b-0da1-4195-8038-166a520a4b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:05.001140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48506f5b-0da1-4195-8038-166a520a4b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:05.016379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48506f5b-0da1-4195-8038-166a520a4b1f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:05.033925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1155/2305] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:37:05.034493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e0432-d505-4e52-962d-96cd46882af4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:05.050764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e0432-d505-4e52-962d-96cd46882af4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:05.078983 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df3d70-f658-4b4f-9e80-9de4d0ae13ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:05.079678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df3d70-f658-4b4f-9e80-9de4d0ae13ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:05.080582 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df3d70-f658-4b4f-9e80-9de4d0ae13ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:05.080973 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df3d70-f658-4b4f-9e80-9de4d0ae13ec admin admin] PATH_INFO: `/v3/projects/1387bc55d4a249638617853ef4b12046/users/940ab95bcc124e7fb037fccf0b95fdf9/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:05.100655 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df3d70-f658-4b4f-9e80-9de4d0ae13ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1387bc55d4a249638617853ef4b12046, user_id=940ab95bcc124e7fb037fccf0b95fdf9, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:05.103461 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df3d70-f658-4b4f-9e80-9de4d0ae13ec admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:05.124268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1151/2306] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:37:05 2022] PUT /identity/v3/projects/1387bc55d4a249638617853ef4b12046/users/940ab95bcc124e7fb037fccf0b95fdf9/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:37:05.128659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48506f5b-0da1-4195-8038-166a520a4b1f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:05.141904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48506f5b-0da1-4195-8038-166a520a4b1f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:05.214438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a07402-6ca7-4536-98a8-e6c6d5cce1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:05.215486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a07402-6ca7-4536-98a8-e6c6d5cce1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:05.215620 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a07402-6ca7-4536-98a8-e6c6d5cce1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:05.215873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a07402-6ca7-4536-98a8-e6c6d5cce1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:05.216525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a07402-6ca7-4536-98a8-e6c6d5cce1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:05.218334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a07402-6ca7-4536-98a8-e6c6d5cce1c1 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:05.235787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df3d70-f658-4b4f-9e80-9de4d0ae13ec admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:05.236783 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1156/2307] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:37:05.247688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df3d70-f658-4b4f-9e80-9de4d0ae13ec admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:05.251612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a07402-6ca7-4536-98a8-e6c6d5cce1c1 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:05.253766 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbbab6-28c6-42bc-aafd-1df2da1e7874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:05.254137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-45a07402-6ca7-4536-98a8-e6c6d5cce1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:05.254441 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbbab6-28c6-42bc-aafd-1df2da1e7874 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:05.254719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbbab6-28c6-42bc-aafd-1df2da1e7874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:05.254993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbbab6-28c6-42bc-aafd-1df2da1e7874 admin admin] PATH_INFO: `/v3/projects/1387bc55d4a249638617853ef4b12046/users/940ab95bcc124e7fb037fccf0b95fdf9/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:05.257099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbbab6-28c6-42bc-aafd-1df2da1e7874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1387bc55d4a249638617853ef4b12046, user_id=940ab95bcc124e7fb037fccf0b95fdf9, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:05.259887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbbab6-28c6-42bc-aafd-1df2da1e7874 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:05.271525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1152/2308] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:37:05 2022] PUT /identity/v3/projects/1387bc55d4a249638617853ef4b12046/users/940ab95bcc124e7fb037fccf0b95fdf9/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:37:05.328864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a07402-6ca7-4536-98a8-e6c6d5cce1c1 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:05.391913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75a693-e8eb-45c1-be34-b7e8138e728c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:05.393165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75a693-e8eb-45c1-be34-b7e8138e728c service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:05.393546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75a693-e8eb-45c1-be34-b7e8138e728c service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:05.393876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75a693-e8eb-45c1-be34-b7e8138e728c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:05.395745 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ec75a693-e8eb-45c1-be34-b7e8138e728c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:05.405825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75a693-e8eb-45c1-be34-b7e8138e728c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:05.407190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75a693-e8eb-45c1-be34-b7e8138e728c service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:05.416901 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1157/2309] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:37:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:37:05.828459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbbab6-28c6-42bc-aafd-1df2da1e7874 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:05.834583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f1dbbab6-28c6-42bc-aafd-1df2da1e7874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:05.981028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbbab6-28c6-42bc-aafd-1df2da1e7874 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:05.991535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5ba41-9223-4655-ad63-b2711f3c94b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:05.993332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5ba41-9223-4655-ad63-b2711f3c94b3 service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:05.993332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5ba41-9223-4655-ad63-b2711f3c94b3 service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:05.993657 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5ba41-9223-4655-ad63-b2711f3c94b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:05.996271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8ac5ba41-9223-4655-ad63-b2711f3c94b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:06.013545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5ba41-9223-4655-ad63-b2711f3c94b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:06.015247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5ba41-9223-4655-ad63-b2711f3c94b3 service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:06.027167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1153/2310] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:37:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:37:07.528425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75a693-e8eb-45c1-be34-b7e8138e728c service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:07.530064 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ec75a693-e8eb-45c1-be34-b7e8138e728c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:07.562886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:37:07.637985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75a693-e8eb-45c1-be34-b7e8138e728c service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:07.644011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ddc36-0675-44a3-b0ea-da9511cc6ec8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:07.644919 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ddc36-0675-44a3-b0ea-da9511cc6ec8 service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:07.645340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ddc36-0675-44a3-b0ea-da9511cc6ec8 service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:07.645758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ddc36-0675-44a3-b0ea-da9511cc6ec8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:07.647718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e67ddc36-0675-44a3-b0ea-da9511cc6ec8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:07.726858 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ddc36-0675-44a3-b0ea-da9511cc6ec8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:07.728829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ddc36-0675-44a3-b0ea-da9511cc6ec8 service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:07.753093 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1158/2311] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:37:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:37:11.996088 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fc3b4-cfa0-4624-b36d-7d9706faf730 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:11.996665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fc3b4-cfa0-4624-b36d-7d9706faf730 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:11.996665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fc3b4-cfa0-4624-b36d-7d9706faf730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:12.028633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f88fc3b4-cfa0-4624-b36d-7d9706faf730 None None] MFA Rules not processed for user `940ab95bcc124e7fb037fccf0b95fdf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:37:12.029906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f88fc3b4-cfa0-4624-b36d-7d9706faf730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:12.108846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1154/2312] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:37:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:37:12.122965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ddc36-0675-44a3-b0ea-da9511cc6ec8 service nova] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:12.135898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ddc36-0675-44a3-b0ea-da9511cc6ec8 service nova] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:12.141782 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86425cee-1db7-4093-b83d-e9f762b99a70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:12.142447 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86425cee-1db7-4093-b83d-e9f762b99a70 service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:12.142728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86425cee-1db7-4093-b83d-e9f762b99a70 service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:12.143002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86425cee-1db7-4093-b83d-e9f762b99a70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:12.203264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86425cee-1db7-4093-b83d-e9f762b99a70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:12.204942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86425cee-1db7-4093-b83d-e9f762b99a70 service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:12.213827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1159/2313] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:37:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:37:22.105439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:INSERT=5,keystone:SELECT=83 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:37:22.209834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=73 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:37:30.129698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fc3b4-cfa0-4624-b36d-7d9706faf730 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:30.131090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f88fc3b4-cfa0-4624-b36d-7d9706faf730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:30.187717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fc3b4-cfa0-4624-b36d-7d9706faf730 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:30.191429 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f6e8e-9775-4901-9ae9-324287b37ff5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:30.192037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f6e8e-9775-4901-9ae9-324287b37ff5 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:30.192318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f6e8e-9775-4901-9ae9-324287b37ff5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:30.192613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f6e8e-9775-4901-9ae9-324287b37ff5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:30.194372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-008f6e8e-9775-4901-9ae9-324287b37ff5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:30.254285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f6e8e-9775-4901-9ae9-324287b37ff5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:30.255722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f6e8e-9775-4901-9ae9-324287b37ff5 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:30.264567 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1155/2314] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:37:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:37:32.998413 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6c607-a5b4-434d-b786-0b849e871168 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:32.998909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6c607-a5b4-434d-b786-0b849e871168 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:32.999015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6c607-a5b4-434d-b786-0b849e871168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:33.018396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b6d6c607-a5b4-434d-b786-0b849e871168 None None] MFA Rules not processed for user `f989b6fded53458b85b39789a32a39e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:37:33.019613 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b6d6c607-a5b4-434d-b786-0b849e871168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:33.026561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1160/2315] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:37:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:37:38.616269 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f6e8e-9775-4901-9ae9-324287b37ff5 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:38.618871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-008f6e8e-9775-4901-9ae9-324287b37ff5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:38.654178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f6e8e-9775-4901-9ae9-324287b37ff5 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:38.664215 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c9a29-8c2c-42ff-a381-3c3172dd7958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:38.665801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c9a29-8c2c-42ff-a381-3c3172dd7958 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:38.666266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c9a29-8c2c-42ff-a381-3c3172dd7958 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:38.667043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c9a29-8c2c-42ff-a381-3c3172dd7958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:38.669485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cc8c9a29-8c2c-42ff-a381-3c3172dd7958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:38.695534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c9a29-8c2c-42ff-a381-3c3172dd7958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:38.699148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c9a29-8c2c-42ff-a381-3c3172dd7958 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:38.716695 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1156/2316] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:37:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:37:38.729543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6c607-a5b4-434d-b786-0b849e871168 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:38.744034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6c607-a5b4-434d-b786-0b849e871168 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:38.750861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7b552-c9c3-40f9-8df5-293febf634d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:38.751989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7b552-c9c3-40f9-8df5-293febf634d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:38.752912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7b552-c9c3-40f9-8df5-293febf634d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:38.753387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7b552-c9c3-40f9-8df5-293febf634d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:38.755562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6ef7b552-c9c3-40f9-8df5-293febf634d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:38.768563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7b552-c9c3-40f9-8df5-293febf634d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:38.770129 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7b552-c9c3-40f9-8df5-293febf634d5 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:38.784442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1161/2317] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:37:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:37:39.535606 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c9a29-8c2c-42ff-a381-3c3172dd7958 service nova] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:39.554498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c9a29-8c2c-42ff-a381-3c3172dd7958 service nova] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:39.565081 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fd098-8f7b-467e-bbc2-ec68c5eefee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:39.566173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fd098-8f7b-467e-bbc2-ec68c5eefee3 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:39.566754 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fd098-8f7b-467e-bbc2-ec68c5eefee3 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:39.567401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fd098-8f7b-467e-bbc2-ec68c5eefee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:39.574803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-203fd098-8f7b-467e-bbc2-ec68c5eefee3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:39.589276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fd098-8f7b-467e-bbc2-ec68c5eefee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:39.591259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fd098-8f7b-467e-bbc2-ec68c5eefee3 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:39.607660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1157/2318] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:37:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:37:48.779013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=9 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:37:49.603351 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=60 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:37:50.473402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7b552-c9c3-40f9-8df5-293febf634d5 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:50.487312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7b552-c9c3-40f9-8df5-293febf634d5 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:50.492980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75152-15a8-437c-b09c-4ddf00ed035b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:50.493583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75152-15a8-437c-b09c-4ddf00ed035b service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:50.493859 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75152-15a8-437c-b09c-4ddf00ed035b service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:50.494130 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75152-15a8-437c-b09c-4ddf00ed035b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:50.496021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-48d75152-15a8-437c-b09c-4ddf00ed035b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:50.509701 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d75152-15a8-437c-b09c-4ddf00ed035b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:50.511365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d75152-15a8-437c-b09c-4ddf00ed035b service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:50.522934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1162/2319] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:37:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:37:51.959900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fd098-8f7b-467e-bbc2-ec68c5eefee3 service nova] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:51.971678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fd098-8f7b-467e-bbc2-ec68c5eefee3 service nova] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:51.977167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a62572-971e-4be5-ae60-b7233deaee1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:51.977760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a62572-971e-4be5-ae60-b7233deaee1a service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:51.978041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a62572-971e-4be5-ae60-b7233deaee1a service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:51.978315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a62572-971e-4be5-ae60-b7233deaee1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:51.979994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b8a62572-971e-4be5-ae60-b7233deaee1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:52.038193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a62572-971e-4be5-ae60-b7233deaee1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:52.039781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a62572-971e-4be5-ae60-b7233deaee1a service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:52.049634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1158/2320] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:37:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:37:56.330495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75152-15a8-437c-b09c-4ddf00ed035b service nova] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:56.339789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75152-15a8-437c-b09c-4ddf00ed035b service nova] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:56.344416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f614995-747b-44c0-9a8a-4f83e8838753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:56.345048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f614995-747b-44c0-9a8a-4f83e8838753 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:56.345330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f614995-747b-44c0-9a8a-4f83e8838753 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:56.345606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f614995-747b-44c0-9a8a-4f83e8838753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:56.354959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f614995-747b-44c0-9a8a-4f83e8838753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:56.356389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f614995-747b-44c0-9a8a-4f83e8838753 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:56.366441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1163/2321] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:37:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:37:56.419587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a62572-971e-4be5-ae60-b7233deaee1a service nova] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:37:56.429662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a62572-971e-4be5-ae60-b7233deaee1a service nova] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:37:56.433392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2c248-8ab2-4825-8eb9-510f7bbb3d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:37:56.434070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2c248-8ab2-4825-8eb9-510f7bbb3d22 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:37:56.434437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2c248-8ab2-4825-8eb9-510f7bbb3d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:37:56.434778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2c248-8ab2-4825-8eb9-510f7bbb3d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:37:56.437251 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-13b2c248-8ab2-4825-8eb9-510f7bbb3d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:37:56.504145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2c248-8ab2-4825-8eb9-510f7bbb3d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:37:56.505484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2c248-8ab2-4825-8eb9-510f7bbb3d22 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:37:56.515625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1159/2322] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:38:00.159477 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f614995-747b-44c0-9a8a-4f83e8838753 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:00.161170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2f614995-747b-44c0-9a8a-4f83e8838753 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:00.177025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f614995-747b-44c0-9a8a-4f83e8838753 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:00.181701 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959843e-20ab-493c-acfe-0c7cd6c953de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:00.182326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959843e-20ab-493c-acfe-0c7cd6c953de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:00.182606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959843e-20ab-493c-acfe-0c7cd6c953de admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:00.182881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959843e-20ab-493c-acfe-0c7cd6c953de admin admin] PATH_INFO: `/v3/users/7936bf23a031418b8ce304aa3bbcf8a2` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:00.183998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959843e-20ab-493c-acfe-0c7cd6c953de admin admin] RBAC: Authorizing `identity:delete_user(user_id=7936bf23a031418b8ce304aa3bbcf8a2)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:00.186838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959843e-20ab-493c-acfe-0c7cd6c953de admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:00.235290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2959843e-20ab-493c-acfe-0c7cd6c953de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7936bf23a031418b8ce304aa3bbcf8a2'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:00.235973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2959843e-20ab-493c-acfe-0c7cd6c953de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7936bf23a031418b8ce304aa3bbcf8a2'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:00.236789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2959843e-20ab-493c-acfe-0c7cd6c953de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7936bf23a031418b8ce304aa3bbcf8a2'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:00.244116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2959843e-20ab-493c-acfe-0c7cd6c953de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7936bf23a031418b8ce304aa3bbcf8a2'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:00.256005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-2959843e-20ab-493c-acfe-0c7cd6c953de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7936bf23a031418b8ce304aa3bbcf8a2'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:00.270229 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1164/2323] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:38:00 2022] DELETE /identity/v3/users/7936bf23a031418b8ce304aa3bbcf8a2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:38:00.449157 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2c248-8ab2-4825-8eb9-510f7bbb3d22 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:00.461069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2c248-8ab2-4825-8eb9-510f7bbb3d22 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:00.522637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c20da4-4679-43a4-b2a8-c7ccf998a780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:00.523303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c20da4-4679-43a4-b2a8-c7ccf998a780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:00.523582 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c20da4-4679-43a4-b2a8-c7ccf998a780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:00.523857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c20da4-4679-43a4-b2a8-c7ccf998a780 admin admin] PATH_INFO: `/v3/projects/b5638f78ab9440479a209d0023e6d0bd` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:00.525009 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c20da4-4679-43a4-b2a8-c7ccf998a780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5638f78ab9440479a209d0023e6d0bd)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:00.526929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c20da4-4679-43a4-b2a8-c7ccf998a780 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:00.565255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:00.577933 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66c20da4-4679-43a4-b2a8-c7ccf998a780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5638f78ab9440479a209d0023e6d0bd', 'request_id': 'req-66c20da4-4679-43a4-b2a8-c7ccf998a780'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:00.583605 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66c20da4-4679-43a4-b2a8-c7ccf998a780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5638f78ab9440479a209d0023e6d0bd', 'request_id': 'req-66c20da4-4679-43a4-b2a8-c7ccf998a780'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:00.589773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66c20da4-4679-43a4-b2a8-c7ccf998a780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5638f78ab9440479a209d0023e6d0bd', 'request_id': 'req-66c20da4-4679-43a4-b2a8-c7ccf998a780'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:00.602871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66c20da4-4679-43a4-b2a8-c7ccf998a780 admin admin] The token cache is being invalidate because project b5638f78ab9440479a209d0023e6d0bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:38:00.603157 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66c20da4-4679-43a4-b2a8-c7ccf998a780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:00.603775 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66c20da4-4679-43a4-b2a8-c7ccf998a780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:00.607974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1160/2324] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:38:00 2022] DELETE /identity/v3/projects/b5638f78ab9440479a209d0023e6d0bd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:00.613127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7b99a-2bd0-4ab9-8913-76289e65fad3 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:00.613530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7b99a-2bd0-4ab9-8913-76289e65fad3 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:00.613811 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7b99a-2bd0-4ab9-8913-76289e65fad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:00.635248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d5f7b99a-2bd0-4ab9-8913-76289e65fad3 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:00.636455 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d5f7b99a-2bd0-4ab9-8913-76289e65fad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:00.696619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1165/2325] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:00.704372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c0fb9-1baf-476c-a42e-7671e22c4104 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:00.704771 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c0fb9-1baf-476c-a42e-7671e22c4104 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:00.705141 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c0fb9-1baf-476c-a42e-7671e22c4104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:00.724427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-299c0fb9-1baf-476c-a42e-7671e22c4104 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:00.725672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-299c0fb9-1baf-476c-a42e-7671e22c4104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:00.734120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1161/2326] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:00.738433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7b99a-2bd0-4ab9-8913-76289e65fad3 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:00.748683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7b99a-2bd0-4ab9-8913-76289e65fad3 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:00.753257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60353af8-bb67-4735-8b38-b4000f53d352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:00.753845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60353af8-bb67-4735-8b38-b4000f53d352 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:00.754123 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60353af8-bb67-4735-8b38-b4000f53d352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:00.754395 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60353af8-bb67-4735-8b38-b4000f53d352 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:00.755035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60353af8-bb67-4735-8b38-b4000f53d352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:38:00.755338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60353af8-bb67-4735-8b38-b4000f53d352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:00.756937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60353af8-bb67-4735-8b38-b4000f53d352 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:00.768838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1166/2327] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:38:00.773102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c0fb9-1baf-476c-a42e-7671e22c4104 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:00.782973 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c0fb9-1baf-476c-a42e-7671e22c4104 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:00.787815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f46db-6426-4d27-8196-e908393e6c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:00.788412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f46db-6426-4d27-8196-e908393e6c5a admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:00.788687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f46db-6426-4d27-8196-e908393e6c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:00.788986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f46db-6426-4d27-8196-e908393e6c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:00.789603 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f46db-6426-4d27-8196-e908393e6c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:00.791533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f46db-6426-4d27-8196-e908393e6c5a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:00.818366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1162/2328] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:38:00 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:00.822732 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60353af8-bb67-4735-8b38-b4000f53d352 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:00.833345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60353af8-bb67-4735-8b38-b4000f53d352 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:00.863206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:00.892449 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a12f1-1769-4ea1-9652-07c1db1045a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:00.893106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a12f1-1769-4ea1-9652-07c1db1045a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:00.893388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a12f1-1769-4ea1-9652-07c1db1045a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:00.893663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a12f1-1769-4ea1-9652-07c1db1045a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:00.894304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a12f1-1769-4ea1-9652-07c1db1045a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:00.896257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a12f1-1769-4ea1-9652-07c1db1045a3 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:00.919466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1167/2329] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:38:00 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:00.923618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f46db-6426-4d27-8196-e908393e6c5a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:00.934504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f46db-6426-4d27-8196-e908393e6c5a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:00.939831 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38684edb-5489-4f9e-9f03-734fe435b306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:00.940458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684edb-5489-4f9e-9f03-734fe435b306 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:00.940724 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684edb-5489-4f9e-9f03-734fe435b306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:00.941029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684edb-5489-4f9e-9f03-734fe435b306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:00.941662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38684edb-5489-4f9e-9f03-734fe435b306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:00.944309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38684edb-5489-4f9e-9f03-734fe435b306 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:00.956517 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1163/2330] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:00.960528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a12f1-1769-4ea1-9652-07c1db1045a3 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:00.971160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a12f1-1769-4ea1-9652-07c1db1045a3 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:00.976215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916cc03-9f60-4a32-9eae-70651b1a742b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:00.976918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916cc03-9f60-4a32-9eae-70651b1a742b admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:00.977265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916cc03-9f60-4a32-9eae-70651b1a742b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:00.977602 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916cc03-9f60-4a32-9eae-70651b1a742b admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:00.978309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916cc03-9f60-4a32-9eae-70651b1a742b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:00.979899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916cc03-9f60-4a32-9eae-70651b1a742b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:00.991220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1168/2331] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:00.995491 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38684edb-5489-4f9e-9f03-734fe435b306 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:01.006171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38684edb-5489-4f9e-9f03-734fe435b306 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:01.010788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe103d-16da-4d06-8cfb-87c0d20202cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:01.011434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe103d-16da-4d06-8cfb-87c0d20202cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:01.011710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe103d-16da-4d06-8cfb-87c0d20202cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:01.011980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe103d-16da-4d06-8cfb-87c0d20202cb admin admin] PATH_INFO: `/v3/projects/2e4928861c8d4e12b9e1e675e299fdce/users/2cd5adf2aba446db8f4d0cb1ed248967/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:01.026476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe103d-16da-4d06-8cfb-87c0d20202cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e4928861c8d4e12b9e1e675e299fdce, user_id=2cd5adf2aba446db8f4d0cb1ed248967, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:01.029866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe103d-16da-4d06-8cfb-87c0d20202cb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:01.042189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1164/2332] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:38:00 2022] PUT /identity/v3/projects/2e4928861c8d4e12b9e1e675e299fdce/users/2cd5adf2aba446db8f4d0cb1ed248967/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:38:01.053816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccdc99-2bbd-4b63-bee3-60cc3b0aa48f None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:01.054134 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccdc99-2bbd-4b63-bee3-60cc3b0aa48f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:01.054413 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccdc99-2bbd-4b63-bee3-60cc3b0aa48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:01.082645 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-e2ccdc99-2bbd-4b63-bee3-60cc3b0aa48f None None] MFA Rules not processed for user `2cd5adf2aba446db8f4d0cb1ed248967`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:01.083787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e2ccdc99-2bbd-4b63-bee3-60cc3b0aa48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:01.146368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1169/2333] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:01.150917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe103d-16da-4d06-8cfb-87c0d20202cb admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:01.212658 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe103d-16da-4d06-8cfb-87c0d20202cb admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:01.217910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f14609-6934-4c6c-9d99-4475ceacd688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:01.218629 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f14609-6934-4c6c-9d99-4475ceacd688 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:01.218917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f14609-6934-4c6c-9d99-4475ceacd688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:01.219193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f14609-6934-4c6c-9d99-4475ceacd688 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:01.219822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f14609-6934-4c6c-9d99-4475ceacd688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:01.221790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f14609-6934-4c6c-9d99-4475ceacd688 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:01.248888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1165/2334] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:38:01 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:01.252872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccdc99-2bbd-4b63-bee3-60cc3b0aa48f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:01.263779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccdc99-2bbd-4b63-bee3-60cc3b0aa48f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:01.319766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87ddc0-ca2e-4e71-8c0e-b844df05463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:01.320634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87ddc0-ca2e-4e71-8c0e-b844df05463d admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:01.320945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87ddc0-ca2e-4e71-8c0e-b844df05463d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:01.321222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87ddc0-ca2e-4e71-8c0e-b844df05463d admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:01.321853 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87ddc0-ca2e-4e71-8c0e-b844df05463d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:01.323763 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87ddc0-ca2e-4e71-8c0e-b844df05463d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:01.345609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1170/2335] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:38:01 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:01.349727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f14609-6934-4c6c-9d99-4475ceacd688 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:01.360361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f14609-6934-4c6c-9d99-4475ceacd688 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:01.364932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f506aa0-12b8-4983-9690-4ae5505fec57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:01.365535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f506aa0-12b8-4983-9690-4ae5505fec57 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:01.365815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f506aa0-12b8-4983-9690-4ae5505fec57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:01.366087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f506aa0-12b8-4983-9690-4ae5505fec57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:01.366719 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f506aa0-12b8-4983-9690-4ae5505fec57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:01.368320 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f506aa0-12b8-4983-9690-4ae5505fec57 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:01.379923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1166/2336] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:01.384421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87ddc0-ca2e-4e71-8c0e-b844df05463d admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:01.395210 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87ddc0-ca2e-4e71-8c0e-b844df05463d admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:01.399761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd98a06-85bb-41ce-96ae-8e2cbeeff28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:01.400405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd98a06-85bb-41ce-96ae-8e2cbeeff28d admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:01.400678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd98a06-85bb-41ce-96ae-8e2cbeeff28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:01.400975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd98a06-85bb-41ce-96ae-8e2cbeeff28d admin admin] PATH_INFO: `/v3/projects/318fe39ea27e46ce9e6bef5fba4826bd/users/4769878ea68a417a83de09804b2eb509/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:01.416266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd98a06-85bb-41ce-96ae-8e2cbeeff28d admin admin] RBAC: Authorizing `identity:create_grant(project_id=318fe39ea27e46ce9e6bef5fba4826bd, user_id=4769878ea68a417a83de09804b2eb509, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:01.419417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd98a06-85bb-41ce-96ae-8e2cbeeff28d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:01.431810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1171/2337] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:38:01 2022] PUT /identity/v3/projects/318fe39ea27e46ce9e6bef5fba4826bd/users/4769878ea68a417a83de09804b2eb509/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:01.436082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f506aa0-12b8-4983-9690-4ae5505fec57 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:01.446525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f506aa0-12b8-4983-9690-4ae5505fec57 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:01.476998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:01.503793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cae048-4815-4662-adb5-4212dfda6f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:01.504402 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cae048-4815-4662-adb5-4212dfda6f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:01.504678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cae048-4815-4662-adb5-4212dfda6f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:01.504983 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cae048-4815-4662-adb5-4212dfda6f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:01.505616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cae048-4815-4662-adb5-4212dfda6f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:01.507203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cae048-4815-4662-adb5-4212dfda6f18 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:01.517482 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1167/2338] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:01.521792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd98a06-85bb-41ce-96ae-8e2cbeeff28d admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:01.533237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd98a06-85bb-41ce-96ae-8e2cbeeff28d admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:01.537859 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d0d23-1b30-4247-8837-32e24b735010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:01.538502 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d0d23-1b30-4247-8837-32e24b735010 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:01.538789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d0d23-1b30-4247-8837-32e24b735010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:01.539058 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d0d23-1b30-4247-8837-32e24b735010 admin admin] PATH_INFO: `/v3/projects/318fe39ea27e46ce9e6bef5fba4826bd/users/4769878ea68a417a83de09804b2eb509/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:01.540991 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d0d23-1b30-4247-8837-32e24b735010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=318fe39ea27e46ce9e6bef5fba4826bd, user_id=4769878ea68a417a83de09804b2eb509, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:01.544713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d0d23-1b30-4247-8837-32e24b735010 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:01.548429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:01.559116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1172/2339] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:38:01 2022] PUT /identity/v3/projects/318fe39ea27e46ce9e6bef5fba4826bd/users/4769878ea68a417a83de09804b2eb509/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:01.569596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbb744-7006-4a85-ab24-53a47db91b8f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:01.569888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbb744-7006-4a85-ab24-53a47db91b8f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:01.570427 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbb744-7006-4a85-ab24-53a47db91b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:01.599191 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-fccbb744-7006-4a85-ab24-53a47db91b8f None None] MFA Rules not processed for user `4769878ea68a417a83de09804b2eb509`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:01.600331 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fccbb744-7006-4a85-ab24-53a47db91b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:01.671799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1168/2340] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:01.682962 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d0d23-1b30-4247-8837-32e24b735010 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:01.684690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-718d0d23-1b30-4247-8837-32e24b735010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:01.741629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d0d23-1b30-4247-8837-32e24b735010 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:01.745306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d85df4-2485-447a-8f8e-cdee770bf134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:01.745900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d85df4-2485-447a-8f8e-cdee770bf134 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:01.746178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d85df4-2485-447a-8f8e-cdee770bf134 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:01.746447 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d85df4-2485-447a-8f8e-cdee770bf134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:01.755817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d85df4-2485-447a-8f8e-cdee770bf134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:01.757271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d85df4-2485-447a-8f8e-cdee770bf134 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:01.767090 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1173/2341] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:01.846903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbb744-7006-4a85-ab24-53a47db91b8f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:01.856607 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbb744-7006-4a85-ab24-53a47db91b8f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:01.860271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a85677-17eb-4858-a908-4f944fc6bccc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:01.860900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a85677-17eb-4858-a908-4f944fc6bccc service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:01.861212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a85677-17eb-4858-a908-4f944fc6bccc service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:01.861466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a85677-17eb-4858-a908-4f944fc6bccc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:01.921801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a85677-17eb-4858-a908-4f944fc6bccc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:01.923265 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a85677-17eb-4858-a908-4f944fc6bccc service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:01.933012 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1169/2342] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:11.763146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:11.900706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d85df4-2485-447a-8f8e-cdee770bf134 service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:11.929067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=76 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:11.966161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d85df4-2485-447a-8f8e-cdee770bf134 service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:11.970942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:11.971571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:11.971848 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:11.972117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] PATH_INFO: `/v3/users/2cd5adf2aba446db8f4d0cb1ed248967` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:11.973264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd5adf2aba446db8f4d0cb1ed248967)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:11.975178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:12.022942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd5adf2aba446db8f4d0cb1ed248967'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:12.023592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd5adf2aba446db8f4d0cb1ed248967'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:12.024210 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd5adf2aba446db8f4d0cb1ed248967'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:12.031162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd5adf2aba446db8f4d0cb1ed248967'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:12.043020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd5adf2aba446db8f4d0cb1ed248967'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:12.057839 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1174/2343] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:38:11 2022] DELETE /identity/v3/users/2cd5adf2aba446db8f4d0cb1ed248967 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:12.063277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30890d3-d33a-4779-9142-6adb6f96c672 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:12.063565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30890d3-d33a-4779-9142-6adb6f96c672 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:12.063841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30890d3-d33a-4779-9142-6adb6f96c672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:12.083501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d30890d3-d33a-4779-9142-6adb6f96c672 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:12.084743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d30890d3-d33a-4779-9142-6adb6f96c672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:12.161901 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1170/2344] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:12.172713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:12.174393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:12.269632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee00694-f3b7-4e55-80f7-d9a06f237b56 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:12.279564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa668d6-a0b3-44da-ae72-396d73fe03b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:12.280836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa668d6-a0b3-44da-ae72-396d73fe03b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:12.281117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa668d6-a0b3-44da-ae72-396d73fe03b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:12.281437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa668d6-a0b3-44da-ae72-396d73fe03b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:12.292846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa668d6-a0b3-44da-ae72-396d73fe03b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:12.294337 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa668d6-a0b3-44da-ae72-396d73fe03b7 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:12.306989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1175/2345] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:12.737648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30890d3-d33a-4779-9142-6adb6f96c672 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:12.739356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d30890d3-d33a-4779-9142-6adb6f96c672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:12.767673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30890d3-d33a-4779-9142-6adb6f96c672 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:12.773655 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:12.774316 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:12.774598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:12.774870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] PATH_INFO: `/v3/projects/2e4928861c8d4e12b9e1e675e299fdce` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:12.776018 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e4928861c8d4e12b9e1e675e299fdce)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:12.778132 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:12.844072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e4928861c8d4e12b9e1e675e299fdce', 'request_id': 'req-acc8413a-058b-4cb0-9314-70fc734f143e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:12.851759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e4928861c8d4e12b9e1e675e299fdce', 'request_id': 'req-acc8413a-058b-4cb0-9314-70fc734f143e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:12.862697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e4928861c8d4e12b9e1e675e299fdce', 'request_id': 'req-acc8413a-058b-4cb0-9314-70fc734f143e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:12.884540 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] The token cache is being invalidate because project 2e4928861c8d4e12b9e1e675e299fdce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:38:12.885227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:12.885736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:12.888643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1171/2346] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:38:12 2022] DELETE /identity/v3/projects/2e4928861c8d4e12b9e1e675e299fdce => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:12.893629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa668d6-a0b3-44da-ae72-396d73fe03b7 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:12.906061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa668d6-a0b3-44da-ae72-396d73fe03b7 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:12.927370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:12.983599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1776dc3-9041-49ae-843d-764b480f420d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:12.983832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1776dc3-9041-49ae-843d-764b480f420d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:12.985119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1776dc3-9041-49ae-843d-764b480f420d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:12.985119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1776dc3-9041-49ae-843d-764b480f420d admin admin] PATH_INFO: `/v3/users/4769878ea68a417a83de09804b2eb509` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:12.987464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1776dc3-9041-49ae-843d-764b480f420d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4769878ea68a417a83de09804b2eb509)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:12.989907 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1776dc3-9041-49ae-843d-764b480f420d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:13.062541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f1776dc3-9041-49ae-843d-764b480f420d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4769878ea68a417a83de09804b2eb509'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:13.063408 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f1776dc3-9041-49ae-843d-764b480f420d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4769878ea68a417a83de09804b2eb509'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:13.064157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f1776dc3-9041-49ae-843d-764b480f420d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4769878ea68a417a83de09804b2eb509'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:13.073203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f1776dc3-9041-49ae-843d-764b480f420d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4769878ea68a417a83de09804b2eb509'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:13.092173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-f1776dc3-9041-49ae-843d-764b480f420d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4769878ea68a417a83de09804b2eb509'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:13.112791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1176/2347] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:38:12 2022] DELETE /identity/v3/users/4769878ea68a417a83de09804b2eb509 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:13.437384 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:13.439228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:13.454826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8413a-058b-4cb0-9314-70fc734f143e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:13.524705 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda09c6-3c29-468d-8920-bcb9d7411ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:13.525452 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda09c6-3c29-468d-8920-bcb9d7411ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:13.525971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda09c6-3c29-468d-8920-bcb9d7411ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:13.526365 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda09c6-3c29-468d-8920-bcb9d7411ea3 admin admin] PATH_INFO: `/v3/projects/318fe39ea27e46ce9e6bef5fba4826bd` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:13.527702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda09c6-3c29-468d-8920-bcb9d7411ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=318fe39ea27e46ce9e6bef5fba4826bd)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:13.529779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda09c6-3c29-468d-8920-bcb9d7411ea3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:13.567770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:13.589910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-edda09c6-3c29-468d-8920-bcb9d7411ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318fe39ea27e46ce9e6bef5fba4826bd', 'request_id': 'req-edda09c6-3c29-468d-8920-bcb9d7411ea3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:13.599393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-edda09c6-3c29-468d-8920-bcb9d7411ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318fe39ea27e46ce9e6bef5fba4826bd', 'request_id': 'req-edda09c6-3c29-468d-8920-bcb9d7411ea3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:13.606832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-edda09c6-3c29-468d-8920-bcb9d7411ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318fe39ea27e46ce9e6bef5fba4826bd', 'request_id': 'req-edda09c6-3c29-468d-8920-bcb9d7411ea3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:13.628811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-edda09c6-3c29-468d-8920-bcb9d7411ea3 admin admin] The token cache is being invalidate because project 318fe39ea27e46ce9e6bef5fba4826bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:38:13.629662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-edda09c6-3c29-468d-8920-bcb9d7411ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:13.630333 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-edda09c6-3c29-468d-8920-bcb9d7411ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:13.632160 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1172/2348] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:38:13 2022] DELETE /identity/v3/projects/318fe39ea27e46ce9e6bef5fba4826bd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:13.636816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4846f-bde9-4a37-9bea-47a7d05a7ecb None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:13.637104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4846f-bde9-4a37-9bea-47a7d05a7ecb None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:13.637437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4846f-bde9-4a37-9bea-47a7d05a7ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:13.669639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-4fd4846f-bde9-4a37-9bea-47a7d05a7ecb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:13.671368 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-4fd4846f-bde9-4a37-9bea-47a7d05a7ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:13.737441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1177/2349] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:13.747451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3923d-d38a-44d8-b4f9-e75e979da6a4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:13.747833 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3923d-d38a-44d8-b4f9-e75e979da6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:13.748218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3923d-d38a-44d8-b4f9-e75e979da6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:13.775212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a4e3923d-d38a-44d8-b4f9-e75e979da6a4 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:13.776630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a4e3923d-d38a-44d8-b4f9-e75e979da6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:13.786938 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1173/2350] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:13.790976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4846f-bde9-4a37-9bea-47a7d05a7ecb None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:13.802469 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4846f-bde9-4a37-9bea-47a7d05a7ecb None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:13.808614 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b8d2b-4369-4bc4-ae62-aa0ece3c6da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:13.809415 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b8d2b-4369-4bc4-ae62-aa0ece3c6da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:13.809731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b8d2b-4369-4bc4-ae62-aa0ece3c6da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:13.810125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b8d2b-4369-4bc4-ae62-aa0ece3c6da6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:13.811777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b8d2b-4369-4bc4-ae62-aa0ece3c6da6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:38:13.812183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b8d2b-4369-4bc4-ae62-aa0ece3c6da6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:13.814677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b8d2b-4369-4bc4-ae62-aa0ece3c6da6 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:13.829691 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1178/2351] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:38:13.834614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3923d-d38a-44d8-b4f9-e75e979da6a4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:13.845584 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3923d-d38a-44d8-b4f9-e75e979da6a4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:13.850768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fc806-bda3-42a4-8701-4f5f8fc64e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:13.851455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fc806-bda3-42a4-8701-4f5f8fc64e06 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:13.851806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fc806-bda3-42a4-8701-4f5f8fc64e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:13.852145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fc806-bda3-42a4-8701-4f5f8fc64e06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:13.852871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588fc806-bda3-42a4-8701-4f5f8fc64e06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:13.855535 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588fc806-bda3-42a4-8701-4f5f8fc64e06 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:13.887860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1174/2352] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:38:13 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:38:13.891924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b8d2b-4369-4bc4-ae62-aa0ece3c6da6 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:13.904072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b8d2b-4369-4bc4-ae62-aa0ece3c6da6 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:13.932505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:13.971512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130a7e9-f3be-4a71-92ae-abb6663a77f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:13.972183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130a7e9-f3be-4a71-92ae-abb6663a77f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:13.972562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130a7e9-f3be-4a71-92ae-abb6663a77f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:13.972878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130a7e9-f3be-4a71-92ae-abb6663a77f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:13.973522 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130a7e9-f3be-4a71-92ae-abb6663a77f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:13.975454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130a7e9-f3be-4a71-92ae-abb6663a77f8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:14.007389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1179/2353] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:38:13 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:38:14.011363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fc806-bda3-42a4-8701-4f5f8fc64e06 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:14.023321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fc806-bda3-42a4-8701-4f5f8fc64e06 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:14.028267 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae69ff-22c4-46a0-a592-42bfd2d6a0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:14.029022 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae69ff-22c4-46a0-a592-42bfd2d6a0db admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:14.029425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae69ff-22c4-46a0-a592-42bfd2d6a0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:14.030338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae69ff-22c4-46a0-a592-42bfd2d6a0db admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:14.031082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae69ff-22c4-46a0-a592-42bfd2d6a0db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:14.033009 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae69ff-22c4-46a0-a592-42bfd2d6a0db admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:14.044615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1175/2354] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:38:14.048618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130a7e9-f3be-4a71-92ae-abb6663a77f8 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:14.059521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130a7e9-f3be-4a71-92ae-abb6663a77f8 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:14.064695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419dc9d-e81c-4d80-91ee-10f934f278e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:14.065338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419dc9d-e81c-4d80-91ee-10f934f278e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:14.065618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419dc9d-e81c-4d80-91ee-10f934f278e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:14.065893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419dc9d-e81c-4d80-91ee-10f934f278e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:14.066534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419dc9d-e81c-4d80-91ee-10f934f278e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:14.068158 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419dc9d-e81c-4d80-91ee-10f934f278e3 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:14.082551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1180/2355] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:14.086454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae69ff-22c4-46a0-a592-42bfd2d6a0db admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:14.099130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae69ff-22c4-46a0-a592-42bfd2d6a0db admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:14.104514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba883879-ee82-4472-bb63-6cf7ff621b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:14.105738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba883879-ee82-4472-bb63-6cf7ff621b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:14.105738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba883879-ee82-4472-bb63-6cf7ff621b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:14.106069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba883879-ee82-4472-bb63-6cf7ff621b4a admin admin] PATH_INFO: `/v3/projects/1545199a87af410db36dd5ffa96941f8/users/2f14fb15a64f489684a7f2479473708e/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:14.122319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba883879-ee82-4472-bb63-6cf7ff621b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1545199a87af410db36dd5ffa96941f8, user_id=2f14fb15a64f489684a7f2479473708e, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:14.125357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba883879-ee82-4472-bb63-6cf7ff621b4a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:14.141501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1176/2356] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:38:14 2022] PUT /identity/v3/projects/1545199a87af410db36dd5ffa96941f8/users/2f14fb15a64f489684a7f2479473708e/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:14.151551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e4185-28e5-40bb-a087-1958d9a9059b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:14.151844 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e4185-28e5-40bb-a087-1958d9a9059b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:14.152120 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e4185-28e5-40bb-a087-1958d9a9059b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:14.182265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-1c2e4185-28e5-40bb-a087-1958d9a9059b None None] MFA Rules not processed for user `2f14fb15a64f489684a7f2479473708e`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:14.183608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-1c2e4185-28e5-40bb-a087-1958d9a9059b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:14.281414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1181/2357] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:14.286967 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba883879-ee82-4472-bb63-6cf7ff621b4a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:14.394586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba883879-ee82-4472-bb63-6cf7ff621b4a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:14.400267 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b12646-a6a0-4770-a21a-7701199f0419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:14.400978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b12646-a6a0-4770-a21a-7701199f0419 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:14.401263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b12646-a6a0-4770-a21a-7701199f0419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:14.401596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b12646-a6a0-4770-a21a-7701199f0419 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:14.402236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b12646-a6a0-4770-a21a-7701199f0419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:14.404329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b12646-a6a0-4770-a21a-7701199f0419 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:14.438855 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1177/2358] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:38:14 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:14.444068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e4185-28e5-40bb-a087-1958d9a9059b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:14.455891 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e4185-28e5-40bb-a087-1958d9a9059b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:14.546723 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98549a1e-b682-4c92-8ce2-51e695a6f1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:14.547429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98549a1e-b682-4c92-8ce2-51e695a6f1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:14.547988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98549a1e-b682-4c92-8ce2-51e695a6f1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:14.548830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98549a1e-b682-4c92-8ce2-51e695a6f1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:14.549678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98549a1e-b682-4c92-8ce2-51e695a6f1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:14.558377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98549a1e-b682-4c92-8ce2-51e695a6f1d0 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:14.600325 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1182/2359] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:38:14 2022] POST /identity/v3/users => generated 299 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:14.604434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b12646-a6a0-4770-a21a-7701199f0419 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:14.618518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b12646-a6a0-4770-a21a-7701199f0419 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:14.625229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7a65d-32a4-4b15-bf16-c8cd0ab5c3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:14.625833 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7a65d-32a4-4b15-bf16-c8cd0ab5c3ef admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:14.626299 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7a65d-32a4-4b15-bf16-c8cd0ab5c3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:14.626731 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7a65d-32a4-4b15-bf16-c8cd0ab5c3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:14.627534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7a65d-32a4-4b15-bf16-c8cd0ab5c3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:14.629388 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7a65d-32a4-4b15-bf16-c8cd0ab5c3ef admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:14.643324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1178/2360] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:14.647837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98549a1e-b682-4c92-8ce2-51e695a6f1d0 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:14.663867 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98549a1e-b682-4c92-8ce2-51e695a6f1d0 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:14.668015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537a7c4-e54f-45fe-b196-29d1ddb53e6d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:14.669707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537a7c4-e54f-45fe-b196-29d1ddb53e6d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:14.671543 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537a7c4-e54f-45fe-b196-29d1ddb53e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:14.672856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03d253-82da-472e-bd0b-5802f026117f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:14.673755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03d253-82da-472e-bd0b-5802f026117f admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:14.674472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03d253-82da-472e-bd0b-5802f026117f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:14.674827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03d253-82da-472e-bd0b-5802f026117f admin admin] PATH_INFO: `/v3/projects/b977b30a443a413593abc46031507fd6/users/4ecaf11ce21f48cd9e8199302d3b8b84/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:14.696742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03d253-82da-472e-bd0b-5802f026117f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b977b30a443a413593abc46031507fd6, user_id=4ecaf11ce21f48cd9e8199302d3b8b84, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:14.702810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03d253-82da-472e-bd0b-5802f026117f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:14.707895 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5537a7c4-e54f-45fe-b196-29d1ddb53e6d None None] MFA Rules not processed for user `d55dc52ab80642da90f05e74092ffdf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:14.709457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5537a7c4-e54f-45fe-b196-29d1ddb53e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:14.728353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1183/2361] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:38:14 2022] PUT /identity/v3/projects/b977b30a443a413593abc46031507fd6/users/4ecaf11ce21f48cd9e8199302d3b8b84/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:38:14.730735 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03d253-82da-472e-bd0b-5802f026117f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:14.739897 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:14.748955 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:14.797799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1179/2362] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:14.810370 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03d253-82da-472e-bd0b-5802f026117f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:14.816324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c13558-6a16-4fe8-a080-00edb9cc58d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:14.817157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c13558-6a16-4fe8-a080-00edb9cc58d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:14.817505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c13558-6a16-4fe8-a080-00edb9cc58d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:14.817841 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c13558-6a16-4fe8-a080-00edb9cc58d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:14.819111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c13558-6a16-4fe8-a080-00edb9cc58d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:14.821312 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c13558-6a16-4fe8-a080-00edb9cc58d1 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:14.839059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1184/2363] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:14.840198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537a7c4-e54f-45fe-b196-29d1ddb53e6d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:14.856586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537a7c4-e54f-45fe-b196-29d1ddb53e6d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:14.862390 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29035c6f-bddd-48be-a22a-41465bb779f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:14.863064 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29035c6f-bddd-48be-a22a-41465bb779f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:14.863371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29035c6f-bddd-48be-a22a-41465bb779f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:14.863870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29035c6f-bddd-48be-a22a-41465bb779f9 admin admin] PATH_INFO: `/v3/projects/b977b30a443a413593abc46031507fd6/users/4ecaf11ce21f48cd9e8199302d3b8b84/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:14.865843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29035c6f-bddd-48be-a22a-41465bb779f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b977b30a443a413593abc46031507fd6, user_id=4ecaf11ce21f48cd9e8199302d3b8b84, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:14.868553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29035c6f-bddd-48be-a22a-41465bb779f9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:14.887977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1180/2364] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:38:14 2022] PUT /identity/v3/projects/b977b30a443a413593abc46031507fd6/users/4ecaf11ce21f48cd9e8199302d3b8b84/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:38:14.894097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4753e-d049-48fe-aab6-af13edd0daf7 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:14.894609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4753e-d049-48fe-aab6-af13edd0daf7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:14.895251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4753e-d049-48fe-aab6-af13edd0daf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:14.931647 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-32a4753e-d049-48fe-aab6-af13edd0daf7 None None] MFA Rules not processed for user `4ecaf11ce21f48cd9e8199302d3b8b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:14.933044 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-32a4753e-d049-48fe-aab6-af13edd0daf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:15.025456 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1185/2365] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:15.040193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29035c6f-bddd-48be-a22a-41465bb779f9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:15.043591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-29035c6f-bddd-48be-a22a-41465bb779f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:15.157153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29035c6f-bddd-48be-a22a-41465bb779f9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:15.161796 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4e4a8-95ba-4ae4-bd31-4503ceb7d3e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:15.162554 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4e4a8-95ba-4ae4-bd31-4503ceb7d3e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:15.162944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4e4a8-95ba-4ae4-bd31-4503ceb7d3e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:15.163559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4e4a8-95ba-4ae4-bd31-4503ceb7d3e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:15.183770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4e4a8-95ba-4ae4-bd31-4503ceb7d3e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:15.185189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4e4a8-95ba-4ae4-bd31-4503ceb7d3e5 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:15.203715 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1181/2366] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:15.233551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4753e-d049-48fe-aab6-af13edd0daf7 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:15.306599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4753e-d049-48fe-aab6-af13edd0daf7 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:15.312402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:15.313655 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:15.313655 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:15.313655 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] PATH_INFO: `/v3/users/2f14fb15a64f489684a7f2479473708e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:15.314836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f14fb15a64f489684a7f2479473708e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:15.316865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:15.347347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:15.374126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f14fb15a64f489684a7f2479473708e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:15.375154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f14fb15a64f489684a7f2479473708e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:15.376500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f14fb15a64f489684a7f2479473708e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:15.385125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f14fb15a64f489684a7f2479473708e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:15.403577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f14fb15a64f489684a7f2479473708e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:15.420974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1186/2367] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:38:15 2022] DELETE /identity/v3/users/2f14fb15a64f489684a7f2479473708e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:15.424112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534f994-c1ce-43ec-bca7-43e58dbf8055 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:15.424633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534f994-c1ce-43ec-bca7-43e58dbf8055 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:15.425228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534f994-c1ce-43ec-bca7-43e58dbf8055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:15.449520 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9534f994-c1ce-43ec-bca7-43e58dbf8055 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:15.452135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9534f994-c1ce-43ec-bca7-43e58dbf8055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:15.540847 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1182/2368] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:15.549536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:15.551105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:15.658484 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933043b-c0ed-4e64-85eb-5a060b1473e6 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:15.663061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3244e8-03d8-4401-920e-b5d2909d6b10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:15.663959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3244e8-03d8-4401-920e-b5d2909d6b10 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:15.664407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3244e8-03d8-4401-920e-b5d2909d6b10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:15.664984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3244e8-03d8-4401-920e-b5d2909d6b10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:15.677251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3244e8-03d8-4401-920e-b5d2909d6b10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:15.679126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3244e8-03d8-4401-920e-b5d2909d6b10 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:15.701103 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1187/2369] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:16.010861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534f994-c1ce-43ec-bca7-43e58dbf8055 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:16.012326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9534f994-c1ce-43ec-bca7-43e58dbf8055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:16.031283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534f994-c1ce-43ec-bca7-43e58dbf8055 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:16.053785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:16.054467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:16.054712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:16.055041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] PATH_INFO: `/v3/projects/1545199a87af410db36dd5ffa96941f8` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:16.056629 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1545199a87af410db36dd5ffa96941f8)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:16.058903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:16.074563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae424c-a7dc-4163-becf-ecc46a6d4fd1 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:16.075056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae424c-a7dc-4163-becf-ecc46a6d4fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:16.075399 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae424c-a7dc-4163-becf-ecc46a6d4fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:16.096285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-5dae424c-a7dc-4163-becf-ecc46a6d4fd1 None None] MFA Rules not processed for user `9b0455ae3ec94c929c940df6a0f6687d`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:16.097705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5dae424c-a7dc-4163-becf-ecc46a6d4fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:16.125734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1545199a87af410db36dd5ffa96941f8', 'request_id': 'req-7a6e0db1-6563-48b1-85d1-ed651fb81026'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.129572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:16.131911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1545199a87af410db36dd5ffa96941f8', 'request_id': 'req-7a6e0db1-6563-48b1-85d1-ed651fb81026'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.140166 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1545199a87af410db36dd5ffa96941f8', 'request_id': 'req-7a6e0db1-6563-48b1-85d1-ed651fb81026'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.153798 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] The token cache is being invalidate because project 1545199a87af410db36dd5ffa96941f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:38:16.154101 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.155238 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.156728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1183/2370] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:38:16 2022] DELETE /identity/v3/projects/1545199a87af410db36dd5ffa96941f8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:16.159693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1188/2371] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:38:16.160594 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:16.161967 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:16.240098 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e0db1-6563-48b1-85d1-ed651fb81026 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:16.245973 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5172c23-65cb-4306-aa2b-2ed37bcf57a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:16.246698 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5172c23-65cb-4306-aa2b-2ed37bcf57a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:16.247054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5172c23-65cb-4306-aa2b-2ed37bcf57a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:16.247337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5172c23-65cb-4306-aa2b-2ed37bcf57a0 admin admin] PATH_INFO: `/v3/users/4ecaf11ce21f48cd9e8199302d3b8b84` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:16.248630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5172c23-65cb-4306-aa2b-2ed37bcf57a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ecaf11ce21f48cd9e8199302d3b8b84)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:16.251620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5172c23-65cb-4306-aa2b-2ed37bcf57a0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:16.313908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c5172c23-65cb-4306-aa2b-2ed37bcf57a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ecaf11ce21f48cd9e8199302d3b8b84'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.314581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c5172c23-65cb-4306-aa2b-2ed37bcf57a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ecaf11ce21f48cd9e8199302d3b8b84'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.315216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c5172c23-65cb-4306-aa2b-2ed37bcf57a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ecaf11ce21f48cd9e8199302d3b8b84'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.328347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c5172c23-65cb-4306-aa2b-2ed37bcf57a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ecaf11ce21f48cd9e8199302d3b8b84'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.335320 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c5172c23-65cb-4306-aa2b-2ed37bcf57a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ecaf11ce21f48cd9e8199302d3b8b84'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.349072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1184/2372] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:38:16 2022] DELETE /identity/v3/users/4ecaf11ce21f48cd9e8199302d3b8b84 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:16.623092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae424c-a7dc-4163-becf-ecc46a6d4fd1 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:16.644927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae424c-a7dc-4163-becf-ecc46a6d4fd1 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:16.732980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:16.754357 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d6eeb-6650-4267-b68a-6b487670e780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:16.754731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d6eeb-6650-4267-b68a-6b487670e780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:16.755363 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d6eeb-6650-4267-b68a-6b487670e780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:16.755648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d6eeb-6650-4267-b68a-6b487670e780 admin admin] PATH_INFO: `/v3/projects/b977b30a443a413593abc46031507fd6` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:16.757083 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d6eeb-6650-4267-b68a-6b487670e780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b977b30a443a413593abc46031507fd6)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:16.759037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d6eeb-6650-4267-b68a-6b487670e780 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:16.820664 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ee3d6eeb-6650-4267-b68a-6b487670e780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b977b30a443a413593abc46031507fd6', 'request_id': 'req-ee3d6eeb-6650-4267-b68a-6b487670e780'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.828566 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ee3d6eeb-6650-4267-b68a-6b487670e780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b977b30a443a413593abc46031507fd6', 'request_id': 'req-ee3d6eeb-6650-4267-b68a-6b487670e780'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.834750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ee3d6eeb-6650-4267-b68a-6b487670e780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b977b30a443a413593abc46031507fd6', 'request_id': 'req-ee3d6eeb-6650-4267-b68a-6b487670e780'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.872243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ee3d6eeb-6650-4267-b68a-6b487670e780 admin admin] The token cache is being invalidate because project b977b30a443a413593abc46031507fd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:38:16.872628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ee3d6eeb-6650-4267-b68a-6b487670e780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.874989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ee3d6eeb-6650-4267-b68a-6b487670e780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:16.880097 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1189/2373] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:38:16 2022] DELETE /identity/v3/projects/b977b30a443a413593abc46031507fd6 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:16.882072 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d92193-c9fd-4c99-9c30-a02444aa4f1a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:16.882517 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d92193-c9fd-4c99-9c30-a02444aa4f1a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:16.882640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d92193-c9fd-4c99-9c30-a02444aa4f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:16.946618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d3d92193-c9fd-4c99-9c30-a02444aa4f1a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:16.947729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d3d92193-c9fd-4c99-9c30-a02444aa4f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:17.034922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1185/2374] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:17.043729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a38567-ebee-4b19-b61c-f87eaf100036 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.044077 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a38567-ebee-4b19-b61c-f87eaf100036 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.044440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a38567-ebee-4b19-b61c-f87eaf100036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.065672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f3a38567-ebee-4b19-b61c-f87eaf100036 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:17.066861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f3a38567-ebee-4b19-b61c-f87eaf100036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:17.075142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1190/2375] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:17.079635 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d92193-c9fd-4c99-9c30-a02444aa4f1a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:17.085058 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:17.091525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d92193-c9fd-4c99-9c30-a02444aa4f1a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:17.097552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897b150-c5a9-4af8-bb3e-c5a0a4c1d7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:17.098191 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897b150-c5a9-4af8-bb3e-c5a0a4c1d7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.098467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897b150-c5a9-4af8-bb3e-c5a0a4c1d7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.098738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897b150-c5a9-4af8-bb3e-c5a0a4c1d7e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.099381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897b150-c5a9-4af8-bb3e-c5a0a4c1d7e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:38:17.099681 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897b150-c5a9-4af8-bb3e-c5a0a4c1d7e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:17.101339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897b150-c5a9-4af8-bb3e-c5a0a4c1d7e9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:17.114539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1186/2376] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:38:17.118837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a38567-ebee-4b19-b61c-f87eaf100036 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:17.129178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a38567-ebee-4b19-b61c-f87eaf100036 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:17.134162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9af722-9c0e-40cf-8920-e208daf6b0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:17.134764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9af722-9c0e-40cf-8920-e208daf6b0af admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.135036 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9af722-9c0e-40cf-8920-e208daf6b0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.135316 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9af722-9c0e-40cf-8920-e208daf6b0af admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.135932 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9af722-9c0e-40cf-8920-e208daf6b0af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:17.137861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9af722-9c0e-40cf-8920-e208daf6b0af admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:17.165522 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1191/2377] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:38:17 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:17.169718 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897b150-c5a9-4af8-bb3e-c5a0a4c1d7e9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:17.182864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897b150-c5a9-4af8-bb3e-c5a0a4c1d7e9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:17.257896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a802da-f1d0-4302-a1bb-811e84e8e92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:17.258550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a802da-f1d0-4302-a1bb-811e84e8e92f admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.258891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a802da-f1d0-4302-a1bb-811e84e8e92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.259183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a802da-f1d0-4302-a1bb-811e84e8e92f admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.259811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a802da-f1d0-4302-a1bb-811e84e8e92f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:17.261796 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a802da-f1d0-4302-a1bb-811e84e8e92f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:17.287236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1187/2378] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:38:17 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:17.291414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9af722-9c0e-40cf-8920-e208daf6b0af admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:17.303956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9af722-9c0e-40cf-8920-e208daf6b0af admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:17.310498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa7232-18f8-4c35-97f5-d146247e3558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:17.311541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa7232-18f8-4c35-97f5-d146247e3558 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.311828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa7232-18f8-4c35-97f5-d146247e3558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.312099 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa7232-18f8-4c35-97f5-d146247e3558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.312833 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa7232-18f8-4c35-97f5-d146247e3558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:17.314423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa7232-18f8-4c35-97f5-d146247e3558 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:17.325636 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1192/2379] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:17.329749 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a802da-f1d0-4302-a1bb-811e84e8e92f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:17.341622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a802da-f1d0-4302-a1bb-811e84e8e92f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:17.348227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc0036-53e7-4a2e-8f62-d1847759d99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:17.349053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc0036-53e7-4a2e-8f62-d1847759d99a admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.349350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc0036-53e7-4a2e-8f62-d1847759d99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.349613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc0036-53e7-4a2e-8f62-d1847759d99a admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.350263 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc0036-53e7-4a2e-8f62-d1847759d99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:17.352891 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc0036-53e7-4a2e-8f62-d1847759d99a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:17.366466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1188/2380] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:17.370531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa7232-18f8-4c35-97f5-d146247e3558 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:17.381303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa7232-18f8-4c35-97f5-d146247e3558 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:17.385977 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276c6ae-6ff7-4b77-a6b9-cdd279cd7e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:17.386632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276c6ae-6ff7-4b77-a6b9-cdd279cd7e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.386908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276c6ae-6ff7-4b77-a6b9-cdd279cd7e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.387227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276c6ae-6ff7-4b77-a6b9-cdd279cd7e85 admin admin] PATH_INFO: `/v3/projects/5b633eff70024e4e8274490b39aa6b4c/users/e7de0d6325294646a676a7510bd5adef/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.402755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276c6ae-6ff7-4b77-a6b9-cdd279cd7e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b633eff70024e4e8274490b39aa6b4c, user_id=e7de0d6325294646a676a7510bd5adef, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:17.406058 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276c6ae-6ff7-4b77-a6b9-cdd279cd7e85 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:17.421739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1193/2381] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:38:17 2022] PUT /identity/v3/projects/5b633eff70024e4e8274490b39aa6b4c/users/e7de0d6325294646a676a7510bd5adef/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:17.432713 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba9dbe-e5e8-4242-a847-b4540e41f128 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.433094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba9dbe-e5e8-4242-a847-b4540e41f128 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.433459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba9dbe-e5e8-4242-a847-b4540e41f128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.471935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ebba9dbe-e5e8-4242-a847-b4540e41f128 None None] MFA Rules not processed for user `e7de0d6325294646a676a7510bd5adef`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:17.473411 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ebba9dbe-e5e8-4242-a847-b4540e41f128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:17.551007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1189/2382] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:17 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:17.555434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276c6ae-6ff7-4b77-a6b9-cdd279cd7e85 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:17.635060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276c6ae-6ff7-4b77-a6b9-cdd279cd7e85 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:17.645507 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34c6cf-55fc-4792-8d5e-f95cdbdf8287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:17.646154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34c6cf-55fc-4792-8d5e-f95cdbdf8287 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.646437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34c6cf-55fc-4792-8d5e-f95cdbdf8287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.646712 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34c6cf-55fc-4792-8d5e-f95cdbdf8287 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.647415 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34c6cf-55fc-4792-8d5e-f95cdbdf8287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:17.649550 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34c6cf-55fc-4792-8d5e-f95cdbdf8287 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:17.683164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1194/2383] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:38:17 2022] POST /identity/v3/projects => generated 399 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:17.684708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba9dbe-e5e8-4242-a847-b4540e41f128 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:17.696393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba9dbe-e5e8-4242-a847-b4540e41f128 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:17.749326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:17.756155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca6818-ef03-463a-baec-5e36fe9e5f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:17.757099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca6818-ef03-463a-baec-5e36fe9e5f92 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.757594 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca6818-ef03-463a-baec-5e36fe9e5f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.758622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca6818-ef03-463a-baec-5e36fe9e5f92 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.758622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca6818-ef03-463a-baec-5e36fe9e5f92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:17.760619 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca6818-ef03-463a-baec-5e36fe9e5f92 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:17.785777 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1190/2384] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:38:17 2022] POST /identity/v3/users => generated 306 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:17.789739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34c6cf-55fc-4792-8d5e-f95cdbdf8287 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:17.795949 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:17.802438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34c6cf-55fc-4792-8d5e-f95cdbdf8287 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:17.813173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334eca1-029e-4d86-b87d-95aac638b2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:17.814619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334eca1-029e-4d86-b87d-95aac638b2f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.814910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334eca1-029e-4d86-b87d-95aac638b2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.815193 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334eca1-029e-4d86-b87d-95aac638b2f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.815843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334eca1-029e-4d86-b87d-95aac638b2f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:17.817687 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334eca1-029e-4d86-b87d-95aac638b2f8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:17.829961 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1195/2385] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:17.833866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca6818-ef03-463a-baec-5e36fe9e5f92 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:17.844681 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca6818-ef03-463a-baec-5e36fe9e5f92 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:17.849693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e9344-6ccb-4b73-b22c-9b7e157c2a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:17.850365 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e9344-6ccb-4b73-b22c-9b7e157c2a93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.850649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e9344-6ccb-4b73-b22c-9b7e157c2a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.850929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e9344-6ccb-4b73-b22c-9b7e157c2a93 admin admin] PATH_INFO: `/v3/projects/03a7f3e602c6496ebe6023c7a9d13b47/users/8272a2a6237741378fcccb9a1a5676b1/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.871482 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e9344-6ccb-4b73-b22c-9b7e157c2a93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a7f3e602c6496ebe6023c7a9d13b47, user_id=8272a2a6237741378fcccb9a1a5676b1, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:17.875304 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e9344-6ccb-4b73-b22c-9b7e157c2a93 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:17.892152 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1191/2386] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:38:17 2022] PUT /identity/v3/projects/03a7f3e602c6496ebe6023c7a9d13b47/users/8272a2a6237741378fcccb9a1a5676b1/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:17.896323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334eca1-029e-4d86-b87d-95aac638b2f8 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:17.907366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334eca1-029e-4d86-b87d-95aac638b2f8 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:17.978887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518d30e-baec-455d-b86d-b38b42e0c7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:17.979440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518d30e-baec-455d-b86d-b38b42e0c7df admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:17.980599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518d30e-baec-455d-b86d-b38b42e0c7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:17.980599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518d30e-baec-455d-b86d-b38b42e0c7df admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:17.980986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518d30e-baec-455d-b86d-b38b42e0c7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:17.983963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518d30e-baec-455d-b86d-b38b42e0c7df admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:18.000943 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1196/2387] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:18.004378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e9344-6ccb-4b73-b22c-9b7e157c2a93 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:18.020268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e9344-6ccb-4b73-b22c-9b7e157c2a93 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:18.031435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482446f-54c8-4449-9ad4-105ba40ee2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:18.032253 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482446f-54c8-4449-9ad4-105ba40ee2c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:18.032631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482446f-54c8-4449-9ad4-105ba40ee2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:18.033208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482446f-54c8-4449-9ad4-105ba40ee2c8 admin admin] PATH_INFO: `/v3/projects/03a7f3e602c6496ebe6023c7a9d13b47/users/8272a2a6237741378fcccb9a1a5676b1/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:18.035484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482446f-54c8-4449-9ad4-105ba40ee2c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a7f3e602c6496ebe6023c7a9d13b47, user_id=8272a2a6237741378fcccb9a1a5676b1, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:18.039464 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482446f-54c8-4449-9ad4-105ba40ee2c8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:18.065085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1192/2388] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:38:18 2022] PUT /identity/v3/projects/03a7f3e602c6496ebe6023c7a9d13b47/users/8272a2a6237741378fcccb9a1a5676b1/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:18.067305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dceee-ad23-412e-b339-087e719c273b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:18.067569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dceee-ad23-412e-b339-087e719c273b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:18.067870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dceee-ad23-412e-b339-087e719c273b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:18.111365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-e89dceee-ad23-412e-b339-087e719c273b None None] MFA Rules not processed for user `8272a2a6237741378fcccb9a1a5676b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:18.112608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e89dceee-ad23-412e-b339-087e719c273b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:18.298755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1197/2389] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:18 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:18.307825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482446f-54c8-4449-9ad4-105ba40ee2c8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:18.312679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8482446f-54c8-4449-9ad4-105ba40ee2c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:18.465694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482446f-54c8-4449-9ad4-105ba40ee2c8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:18.470417 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e50b8-6a8a-42cc-a8ca-44eda798f9b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:18.471207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e50b8-6a8a-42cc-a8ca-44eda798f9b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:18.471539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e50b8-6a8a-42cc-a8ca-44eda798f9b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:18.471896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e50b8-6a8a-42cc-a8ca-44eda798f9b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:18.488678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e50b8-6a8a-42cc-a8ca-44eda798f9b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:18.490176 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e50b8-6a8a-42cc-a8ca-44eda798f9b8 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:18.500663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1193/2390] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:18 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:38:18.578485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dceee-ad23-412e-b339-087e719c273b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:18.587918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dceee-ad23-412e-b339-087e719c273b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:18.591896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb935cd-ba49-46d9-84e3-2cfb4acc78e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:18.592510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb935cd-ba49-46d9-84e3-2cfb4acc78e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:18.592884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb935cd-ba49-46d9-84e3-2cfb4acc78e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:18.593161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb935cd-ba49-46d9-84e3-2cfb4acc78e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:18.667619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb935cd-ba49-46d9-84e3-2cfb4acc78e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:18.669737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb935cd-ba49-46d9-84e3-2cfb4acc78e6 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:18.680555 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1198/2391] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:22.849696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e50b8-6a8a-42cc-a8ca-44eda798f9b8 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:22.915321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e50b8-6a8a-42cc-a8ca-44eda798f9b8 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:22.920767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:22.921436 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:22.921720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:22.921994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] PATH_INFO: `/v3/users/e7de0d6325294646a676a7510bd5adef` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:22.923275 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7de0d6325294646a676a7510bd5adef)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:22.925280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:22.975545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7de0d6325294646a676a7510bd5adef'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:22.976204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7de0d6325294646a676a7510bd5adef'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:22.976843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7de0d6325294646a676a7510bd5adef'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:22.982926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:22.993416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7de0d6325294646a676a7510bd5adef'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:23.000415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7de0d6325294646a676a7510bd5adef'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:23.015942 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1194/2392] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:38:22 2022] DELETE /identity/v3/users/e7de0d6325294646a676a7510bd5adef => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:23.020400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c7d64-7d81-4c49-a65b-9e505a84076f None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:23.020878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c7d64-7d81-4c49-a65b-9e505a84076f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:23.021007 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c7d64-7d81-4c49-a65b-9e505a84076f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:23.040366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-703c7d64-7d81-4c49-a65b-9e505a84076f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:23.041630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-703c7d64-7d81-4c49-a65b-9e505a84076f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:23.046950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:23.105347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1199/2393] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:23.115741 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:23.117205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:23.174593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e7145-f5d4-4e3c-bed9-873c3251b3a8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:23.178139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae5459-46bf-4a69-9e93-af8880d6cdf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:23.178803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae5459-46bf-4a69-9e93-af8880d6cdf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:23.179084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae5459-46bf-4a69-9e93-af8880d6cdf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:23.179355 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae5459-46bf-4a69-9e93-af8880d6cdf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:23.189544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae5459-46bf-4a69-9e93-af8880d6cdf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:23.192359 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae5459-46bf-4a69-9e93-af8880d6cdf6 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:23.204488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1195/2394] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:23.468021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c7d64-7d81-4c49-a65b-9e505a84076f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:23.469506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-703c7d64-7d81-4c49-a65b-9e505a84076f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:23.483137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c7d64-7d81-4c49-a65b-9e505a84076f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:23.488179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:23.488943 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:23.489289 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:23.489626 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] PATH_INFO: `/v3/projects/5b633eff70024e4e8274490b39aa6b4c` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:23.490803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b633eff70024e4e8274490b39aa6b4c)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:23.492875 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:23.539320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b633eff70024e4e8274490b39aa6b4c', 'request_id': 'req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:23.545950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b633eff70024e4e8274490b39aa6b4c', 'request_id': 'req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:23.551371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b633eff70024e4e8274490b39aa6b4c', 'request_id': 'req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:23.564964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] The token cache is being invalidate because project 5b633eff70024e4e8274490b39aa6b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:38:23.565257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:23.565885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:23.567380 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1200/2395] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:38:23 2022] DELETE /identity/v3/projects/5b633eff70024e4e8274490b39aa6b4c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:23.571665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae5459-46bf-4a69-9e93-af8880d6cdf6 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:23.581726 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae5459-46bf-4a69-9e93-af8880d6cdf6 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:23.641223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccad41-a4f8-4bef-98dc-c8063c494820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:23.641867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccad41-a4f8-4bef-98dc-c8063c494820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:23.642142 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccad41-a4f8-4bef-98dc-c8063c494820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:23.642415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccad41-a4f8-4bef-98dc-c8063c494820 admin admin] PATH_INFO: `/v3/users/8272a2a6237741378fcccb9a1a5676b1` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:23.643556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ccad41-a4f8-4bef-98dc-c8063c494820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8272a2a6237741378fcccb9a1a5676b1)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:23.645593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ccad41-a4f8-4bef-98dc-c8063c494820 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:23.701541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-26ccad41-a4f8-4bef-98dc-c8063c494820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8272a2a6237741378fcccb9a1a5676b1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:23.702475 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-26ccad41-a4f8-4bef-98dc-c8063c494820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8272a2a6237741378fcccb9a1a5676b1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:23.703300 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-26ccad41-a4f8-4bef-98dc-c8063c494820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8272a2a6237741378fcccb9a1a5676b1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:23.718155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-26ccad41-a4f8-4bef-98dc-c8063c494820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8272a2a6237741378fcccb9a1a5676b1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:23.725317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-26ccad41-a4f8-4bef-98dc-c8063c494820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8272a2a6237741378fcccb9a1a5676b1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:23.727842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:23.741761 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1196/2396] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:38:23 2022] DELETE /identity/v3/users/8272a2a6237741378fcccb9a1a5676b1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:24.070214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:24.071733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:24.084880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a25f3c-1e85-4e80-9e8d-38d71f3e6d29 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:24.147295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:24.147932 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:24.148207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:24.148476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e admin admin] PATH_INFO: `/v3/projects/03a7f3e602c6496ebe6023c7a9d13b47` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:24.149800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=03a7f3e602c6496ebe6023c7a9d13b47)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:24.151708 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:24.191716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:24.203350 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03a7f3e602c6496ebe6023c7a9d13b47', 'request_id': 'req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:24.209652 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03a7f3e602c6496ebe6023c7a9d13b47', 'request_id': 'req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:24.215195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03a7f3e602c6496ebe6023c7a9d13b47', 'request_id': 'req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:24.226980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e admin admin] The token cache is being invalidate because project 03a7f3e602c6496ebe6023c7a9d13b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:38:24.227269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:24.227881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-efbfacc7-3254-46ea-8fbd-d72fd3b85f5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:24.229391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1201/2397] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:38:24 2022] DELETE /identity/v3/projects/03a7f3e602c6496ebe6023c7a9d13b47 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:24.234360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01710f9a-c834-4e64-bc2a-a544057baaeb None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:24.234839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01710f9a-c834-4e64-bc2a-a544057baaeb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:24.234948 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01710f9a-c834-4e64-bc2a-a544057baaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:24.255654 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-01710f9a-c834-4e64-bc2a-a544057baaeb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:24.256957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-01710f9a-c834-4e64-bc2a-a544057baaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:24.314760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1197/2398] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:24.323036 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e059bc5-360d-4484-9804-d9bc4a433a02 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:24.323329 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e059bc5-360d-4484-9804-d9bc4a433a02 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:24.323613 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e059bc5-360d-4484-9804-d9bc4a433a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:24.344501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3e059bc5-360d-4484-9804-d9bc4a433a02 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:24.345775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3e059bc5-360d-4484-9804-d9bc4a433a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:24.353781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1202/2399] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:24.358328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01710f9a-c834-4e64-bc2a-a544057baaeb None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:24.368046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01710f9a-c834-4e64-bc2a-a544057baaeb None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:24.373421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c249f-55df-42a5-ac19-e00998b69800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:24.374077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c249f-55df-42a5-ac19-e00998b69800 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:24.374348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c249f-55df-42a5-ac19-e00998b69800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:24.374614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c249f-55df-42a5-ac19-e00998b69800 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:24.375280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c249f-55df-42a5-ac19-e00998b69800 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:38:24.375576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c249f-55df-42a5-ac19-e00998b69800 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:24.377199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c249f-55df-42a5-ac19-e00998b69800 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:24.389329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1198/2400] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:38:24.393573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e059bc5-360d-4484-9804-d9bc4a433a02 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:24.402726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e059bc5-360d-4484-9804-d9bc4a433a02 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:24.407585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41bcbbc-8718-405f-909b-5a8555ec5d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:24.408167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bcbbc-8718-405f-909b-5a8555ec5d28 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:24.408441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bcbbc-8718-405f-909b-5a8555ec5d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:24.408710 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bcbbc-8718-405f-909b-5a8555ec5d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:24.409359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41bcbbc-8718-405f-909b-5a8555ec5d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:24.411379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41bcbbc-8718-405f-909b-5a8555ec5d28 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:24.439018 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1203/2401] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:38:24 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:24.443165 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c249f-55df-42a5-ac19-e00998b69800 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:24.454240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c249f-55df-42a5-ac19-e00998b69800 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:24.511668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37a680-9c86-400c-8959-c40766e6c6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:24.512204 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37a680-9c86-400c-8959-c40766e6c6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:24.512514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37a680-9c86-400c-8959-c40766e6c6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:24.512827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37a680-9c86-400c-8959-c40766e6c6b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:24.514395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37a680-9c86-400c-8959-c40766e6c6b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:24.516333 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37a680-9c86-400c-8959-c40766e6c6b7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:24.538048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1199/2402] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:38:24 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:24.542198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41bcbbc-8718-405f-909b-5a8555ec5d28 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:24.553162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41bcbbc-8718-405f-909b-5a8555ec5d28 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:24.557925 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea451f-95bd-43fd-95e4-cbe773f71575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:24.558516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea451f-95bd-43fd-95e4-cbe773f71575 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:24.558788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea451f-95bd-43fd-95e4-cbe773f71575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:24.559053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea451f-95bd-43fd-95e4-cbe773f71575 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:24.559682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea451f-95bd-43fd-95e4-cbe773f71575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:24.561838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea451f-95bd-43fd-95e4-cbe773f71575 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:24.572941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1204/2403] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:38:24.577452 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37a680-9c86-400c-8959-c40766e6c6b7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:24.589175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37a680-9c86-400c-8959-c40766e6c6b7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:24.594452 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e09887-5c31-44e2-bece-9615ac103e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:24.595135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e09887-5c31-44e2-bece-9615ac103e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:24.595480 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e09887-5c31-44e2-bece-9615ac103e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:24.595816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e09887-5c31-44e2-bece-9615ac103e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:24.596543 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e09887-5c31-44e2-bece-9615ac103e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:24.598307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e09887-5c31-44e2-bece-9615ac103e16 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:24.612922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1200/2404] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:24.615191 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea451f-95bd-43fd-95e4-cbe773f71575 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:24.626432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea451f-95bd-43fd-95e4-cbe773f71575 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:24.632072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1fd70-963c-409a-ac25-e0d269a11a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:24.632725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1fd70-963c-409a-ac25-e0d269a11a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:24.633039 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1fd70-963c-409a-ac25-e0d269a11a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:24.633306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1fd70-963c-409a-ac25-e0d269a11a08 admin admin] PATH_INFO: `/v3/projects/db425b60c27c483aa1eb6c2602921fc9/users/a41254b36c2c49c2be412fb3d11b349e/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:24.652470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1fd70-963c-409a-ac25-e0d269a11a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db425b60c27c483aa1eb6c2602921fc9, user_id=a41254b36c2c49c2be412fb3d11b349e, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:24.656190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1fd70-963c-409a-ac25-e0d269a11a08 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:24.671039 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1205/2405] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:38:24 2022] PUT /identity/v3/projects/db425b60c27c483aa1eb6c2602921fc9/users/a41254b36c2c49c2be412fb3d11b349e/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:38:24.681454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450df31-388c-48c4-8301-3d6246fe68bb None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:24.681742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450df31-388c-48c4-8301-3d6246fe68bb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:24.682017 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450df31-388c-48c4-8301-3d6246fe68bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:24.710946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f450df31-388c-48c4-8301-3d6246fe68bb None None] MFA Rules not processed for user `a41254b36c2c49c2be412fb3d11b349e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:24.712306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f450df31-388c-48c4-8301-3d6246fe68bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:24.767013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:24.791549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1201/2406] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:24.808152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1fd70-963c-409a-ac25-e0d269a11a08 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:24.809940 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-46c1fd70-963c-409a-ac25-e0d269a11a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:24.900915 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1fd70-963c-409a-ac25-e0d269a11a08 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:24.922460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703da61e-ce6d-4123-9e6a-38b823af274f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:24.923290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da61e-ce6d-4123-9e6a-38b823af274f service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:24.923581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da61e-ce6d-4123-9e6a-38b823af274f service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:24.923861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da61e-ce6d-4123-9e6a-38b823af274f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:24.937356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703da61e-ce6d-4123-9e6a-38b823af274f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:24.938811 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703da61e-ce6d-4123-9e6a-38b823af274f service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:24.948369 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1206/2407] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:38:24.971441 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450df31-388c-48c4-8301-3d6246fe68bb None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:25.036550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450df31-388c-48c4-8301-3d6246fe68bb None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:25.041774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:25.042418 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:25.042696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:25.043032 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] PATH_INFO: `/v3/users/a41254b36c2c49c2be412fb3d11b349e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:25.044214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a41254b36c2c49c2be412fb3d11b349e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:25.047256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:25.119020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a41254b36c2c49c2be412fb3d11b349e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:25.119846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a41254b36c2c49c2be412fb3d11b349e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:25.120599 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a41254b36c2c49c2be412fb3d11b349e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:25.139668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a41254b36c2c49c2be412fb3d11b349e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:25.149917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a41254b36c2c49c2be412fb3d11b349e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:25.163512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1202/2408] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:38:24 2022] DELETE /identity/v3/users/a41254b36c2c49c2be412fb3d11b349e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:25.168099 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1071de-5a26-4792-ba93-ed2dac782dca None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:25.168564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1071de-5a26-4792-ba93-ed2dac782dca None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:25.168720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1071de-5a26-4792-ba93-ed2dac782dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:25.188011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ad1071de-5a26-4792-ba93-ed2dac782dca None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:25.189240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ad1071de-5a26-4792-ba93-ed2dac782dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:25.267321 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1207/2409] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:25.278177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:25.280608 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:25.346053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1673a-1c78-4670-9253-687f0bb6d0fc admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:25.349909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca9507-68db-435a-b3b2-ccc5d15329b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:25.350602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca9507-68db-435a-b3b2-ccc5d15329b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:25.350955 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca9507-68db-435a-b3b2-ccc5d15329b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:25.351326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca9507-68db-435a-b3b2-ccc5d15329b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:25.360902 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca9507-68db-435a-b3b2-ccc5d15329b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:25.362353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca9507-68db-435a-b3b2-ccc5d15329b0 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:25.372461 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1203/2410] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:25.650579 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1071de-5a26-4792-ba93-ed2dac782dca None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:25.652098 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ad1071de-5a26-4792-ba93-ed2dac782dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:25.661355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:25.666061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1071de-5a26-4792-ba93-ed2dac782dca None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:25.680165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:25.681048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:25.681444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:25.681931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0 admin admin] PATH_INFO: `/v3/projects/db425b60c27c483aa1eb6c2602921fc9` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:25.683069 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db425b60c27c483aa1eb6c2602921fc9)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:25.685128 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:25.733196 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db425b60c27c483aa1eb6c2602921fc9', 'request_id': 'req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:25.739548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db425b60c27c483aa1eb6c2602921fc9', 'request_id': 'req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:25.745094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db425b60c27c483aa1eb6c2602921fc9', 'request_id': 'req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:25.757530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0 admin admin] The token cache is being invalidate because project db425b60c27c483aa1eb6c2602921fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:38:25.757821 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:25.758437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-990fe71f-d85d-4c4e-aa0e-dfd46918d6d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:25.759914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1208/2411] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:38:25 2022] DELETE /identity/v3/projects/db425b60c27c483aa1eb6c2602921fc9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:25.765360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f6234-1ade-4ba3-8660-754b83785c7c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:25.765652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f6234-1ade-4ba3-8660-754b83785c7c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:25.765931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f6234-1ade-4ba3-8660-754b83785c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:25.786118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9e8f6234-1ade-4ba3-8660-754b83785c7c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:25.787264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9e8f6234-1ade-4ba3-8660-754b83785c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:25.792695 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:25.850076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1204/2412] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:25.859597 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a50127-d94c-449a-af32-36633308dd8a None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:25.859882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a50127-d94c-449a-af32-36633308dd8a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:25.860162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a50127-d94c-449a-af32-36633308dd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:25.879637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d9a50127-d94c-449a-af32-36633308dd8a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:25.880866 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d9a50127-d94c-449a-af32-36633308dd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:25.888695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1209/2413] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:25.893289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f6234-1ade-4ba3-8660-754b83785c7c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:25.903445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f6234-1ade-4ba3-8660-754b83785c7c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:25.908928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749140f3-0ba7-457b-8f6a-35870dc03324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:25.909576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749140f3-0ba7-457b-8f6a-35870dc03324 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:25.909894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749140f3-0ba7-457b-8f6a-35870dc03324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:25.910285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749140f3-0ba7-457b-8f6a-35870dc03324 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:25.911017 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749140f3-0ba7-457b-8f6a-35870dc03324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:38:25.911325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749140f3-0ba7-457b-8f6a-35870dc03324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:25.913290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749140f3-0ba7-457b-8f6a-35870dc03324 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:25.925949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1205/2414] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:38:25.930546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a50127-d94c-449a-af32-36633308dd8a None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:25.941758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a50127-d94c-449a-af32-36633308dd8a None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:25.946757 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651446a-a941-42c8-a1bf-3b9ab37d0e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:25.947360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651446a-a941-42c8-a1bf-3b9ab37d0e79 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:25.947638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651446a-a941-42c8-a1bf-3b9ab37d0e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:25.947910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651446a-a941-42c8-a1bf-3b9ab37d0e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:25.948532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651446a-a941-42c8-a1bf-3b9ab37d0e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:25.950487 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651446a-a941-42c8-a1bf-3b9ab37d0e79 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:25.980682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1210/2415] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:38:25 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:25.985848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749140f3-0ba7-457b-8f6a-35870dc03324 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:25.996044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749140f3-0ba7-457b-8f6a-35870dc03324 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:26.061318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f93eec-267a-4a30-b0ec-c13e21f36973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:26.061853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f93eec-267a-4a30-b0ec-c13e21f36973 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:26.062133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f93eec-267a-4a30-b0ec-c13e21f36973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:26.062408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f93eec-267a-4a30-b0ec-c13e21f36973 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:26.063025 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f93eec-267a-4a30-b0ec-c13e21f36973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:26.064980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f93eec-267a-4a30-b0ec-c13e21f36973 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:26.090411 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1206/2416] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:38:25 2022] POST /identity/v3/users => generated 280 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:38:26.092452 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651446a-a941-42c8-a1bf-3b9ab37d0e79 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:26.103164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651446a-a941-42c8-a1bf-3b9ab37d0e79 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:26.109066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb486d6a-a8fe-476d-9113-dd9e2eb385a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:26.109663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb486d6a-a8fe-476d-9113-dd9e2eb385a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:26.109940 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb486d6a-a8fe-476d-9113-dd9e2eb385a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:26.110205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb486d6a-a8fe-476d-9113-dd9e2eb385a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:26.110830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb486d6a-a8fe-476d-9113-dd9e2eb385a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:26.112441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb486d6a-a8fe-476d-9113-dd9e2eb385a0 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:26.123218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1211/2417] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:38:26.127312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f93eec-267a-4a30-b0ec-c13e21f36973 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:26.137708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f93eec-267a-4a30-b0ec-c13e21f36973 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:26.143445 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e2f75-9bfc-4ece-a466-7fe1f2076bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:26.144048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e2f75-9bfc-4ece-a466-7fe1f2076bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:26.144328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e2f75-9bfc-4ece-a466-7fe1f2076bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:26.144600 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e2f75-9bfc-4ece-a466-7fe1f2076bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:26.145271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e2f75-9bfc-4ece-a466-7fe1f2076bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:26.146862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e2f75-9bfc-4ece-a466-7fe1f2076bf6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:26.159262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1207/2418] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:26.163414 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb486d6a-a8fe-476d-9113-dd9e2eb385a0 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:26.173459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb486d6a-a8fe-476d-9113-dd9e2eb385a0 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:26.178293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d0c12-76dc-43d9-9763-6a606a0763c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:26.178937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d0c12-76dc-43d9-9763-6a606a0763c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:26.179215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d0c12-76dc-43d9-9763-6a606a0763c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:26.179488 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d0c12-76dc-43d9-9763-6a606a0763c2 admin admin] PATH_INFO: `/v3/projects/2f8731675d7540e8af5680231d6fd5f1/users/a97dc3cac40c41488c81e572c15aa020/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:26.196078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d0c12-76dc-43d9-9763-6a606a0763c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8731675d7540e8af5680231d6fd5f1, user_id=a97dc3cac40c41488c81e572c15aa020, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:26.198784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d0c12-76dc-43d9-9763-6a606a0763c2 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:26.212724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1212/2419] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:38:26 2022] PUT /identity/v3/projects/2f8731675d7540e8af5680231d6fd5f1/users/a97dc3cac40c41488c81e572c15aa020/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:26.223500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020a8ec-1cd2-4c3e-b337-83c8ffd2ef24 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:26.223788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020a8ec-1cd2-4c3e-b337-83c8ffd2ef24 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:26.224063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020a8ec-1cd2-4c3e-b337-83c8ffd2ef24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:26.252415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6020a8ec-1cd2-4c3e-b337-83c8ffd2ef24 None None] MFA Rules not processed for user `a97dc3cac40c41488c81e572c15aa020`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:26.253693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6020a8ec-1cd2-4c3e-b337-83c8ffd2ef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:26.322640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1208/2420] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:26.357934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d0c12-76dc-43d9-9763-6a606a0763c2 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:26.359674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e18d0c12-76dc-43d9-9763-6a606a0763c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:26.420634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d0c12-76dc-43d9-9763-6a606a0763c2 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:26.424605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a867b2c-0e51-44d8-a9bb-0dbbbf9abd8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:26.425263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a867b2c-0e51-44d8-a9bb-0dbbbf9abd8d service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:26.425543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a867b2c-0e51-44d8-a9bb-0dbbbf9abd8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:26.425815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a867b2c-0e51-44d8-a9bb-0dbbbf9abd8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:26.434241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a867b2c-0e51-44d8-a9bb-0dbbbf9abd8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:26.435724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a867b2c-0e51-44d8-a9bb-0dbbbf9abd8d service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:26.445293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1213/2421] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:26.460687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020a8ec-1cd2-4c3e-b337-83c8ffd2ef24 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:26.471872 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:26.536906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020a8ec-1cd2-4c3e-b337-83c8ffd2ef24 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:26.541710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:26.542355 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:26.543190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:26.543496 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] PATH_INFO: `/v3/users/a97dc3cac40c41488c81e572c15aa020` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:26.551907 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a97dc3cac40c41488c81e572c15aa020)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:26.553940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:26.606115 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a97dc3cac40c41488c81e572c15aa020'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:26.607538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a97dc3cac40c41488c81e572c15aa020'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:26.608364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a97dc3cac40c41488c81e572c15aa020'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:26.620546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a97dc3cac40c41488c81e572c15aa020'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:26.627475 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a97dc3cac40c41488c81e572c15aa020'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:26.645213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1209/2422] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:38:26 2022] DELETE /identity/v3/users/a97dc3cac40c41488c81e572c15aa020 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:26.650386 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae340571-a4c0-483b-8eb3-a3dc1788df35 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:26.650679 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae340571-a4c0-483b-8eb3-a3dc1788df35 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:26.650954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae340571-a4c0-483b-8eb3-a3dc1788df35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:26.672633 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ae340571-a4c0-483b-8eb3-a3dc1788df35 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:26.673866 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ae340571-a4c0-483b-8eb3-a3dc1788df35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:26.694974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:26.742159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1214/2423] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:38:26.751976 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:26.753416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:26.810690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea85818-f499-4f60-9051-0d82aef6c71b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:26.814259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ca68c-34bf-4655-ac70-d9857d39c241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:26.814859 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca68c-34bf-4655-ac70-d9857d39c241 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:26.815136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca68c-34bf-4655-ac70-d9857d39c241 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:26.815409 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca68c-34bf-4655-ac70-d9857d39c241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:26.824728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ca68c-34bf-4655-ac70-d9857d39c241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:26.826683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ca68c-34bf-4655-ac70-d9857d39c241 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:26.836093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1210/2424] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:27.474072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae340571-a4c0-483b-8eb3-a3dc1788df35 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:27.476646 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ae340571-a4c0-483b-8eb3-a3dc1788df35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:27.492423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae340571-a4c0-483b-8eb3-a3dc1788df35 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:27.497192 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:27.498568 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:27.498871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:27.499223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d admin admin] PATH_INFO: `/v3/projects/2f8731675d7540e8af5680231d6fd5f1` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:27.500648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f8731675d7540e8af5680231d6fd5f1)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:27.502543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:27.559665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f8731675d7540e8af5680231d6fd5f1', 'request_id': 'req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:27.566618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f8731675d7540e8af5680231d6fd5f1', 'request_id': 'req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:27.574160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f8731675d7540e8af5680231d6fd5f1', 'request_id': 'req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:27.590640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d admin admin] The token cache is being invalidate because project 2f8731675d7540e8af5680231d6fd5f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:38:27.590938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:27.591686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6b9ecd17-b4ac-4721-98e5-ccdecc9e4d0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:27.594127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1215/2425] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:38:27 2022] DELETE /identity/v3/projects/2f8731675d7540e8af5680231d6fd5f1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:27.598045 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfab813-1308-4b1a-9b77-61bfd2567143 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:27.598514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfab813-1308-4b1a-9b77-61bfd2567143 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:27.598691 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfab813-1308-4b1a-9b77-61bfd2567143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:27.620581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-cdfab813-1308-4b1a-9b77-61bfd2567143 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:27.621717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cdfab813-1308-4b1a-9b77-61bfd2567143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:27.654277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:27.702822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1211/2426] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:27.711885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f856ed-5d02-43bd-b08a-03aada15b70b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:27.712495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f856ed-5d02-43bd-b08a-03aada15b70b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:27.713245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f856ed-5d02-43bd-b08a-03aada15b70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:27.739623 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b3f856ed-5d02-43bd-b08a-03aada15b70b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:27.741212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b3f856ed-5d02-43bd-b08a-03aada15b70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:27.749970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1216/2427] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:38:27.754257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfab813-1308-4b1a-9b77-61bfd2567143 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:27.764646 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfab813-1308-4b1a-9b77-61bfd2567143 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:27.769278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6a227-e056-41a9-8201-91ba5a9dde43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:27.769884 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6a227-e056-41a9-8201-91ba5a9dde43 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:27.770162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6a227-e056-41a9-8201-91ba5a9dde43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:27.770432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6a227-e056-41a9-8201-91ba5a9dde43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:27.771079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6a227-e056-41a9-8201-91ba5a9dde43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:38:27.771378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6a227-e056-41a9-8201-91ba5a9dde43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:27.773003 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6a227-e056-41a9-8201-91ba5a9dde43 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:27.789432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1212/2428] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:38:27.793836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f856ed-5d02-43bd-b08a-03aada15b70b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:27.803756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f856ed-5d02-43bd-b08a-03aada15b70b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:27.808982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6f73d-790e-4512-bce3-5893303cc8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:27.809595 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6f73d-790e-4512-bce3-5893303cc8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:27.809935 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6f73d-790e-4512-bce3-5893303cc8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:27.810206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6f73d-790e-4512-bce3-5893303cc8c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:27.810832 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6f73d-790e-4512-bce3-5893303cc8c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:27.812797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6f73d-790e-4512-bce3-5893303cc8c8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:27.843983 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1217/2429] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:38:27 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:27.847203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6a227-e056-41a9-8201-91ba5a9dde43 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:27.858342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6a227-e056-41a9-8201-91ba5a9dde43 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:27.918886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63913516-d81b-4dc9-8eb9-0935f9d44cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:27.919981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63913516-d81b-4dc9-8eb9-0935f9d44cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:27.920523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63913516-d81b-4dc9-8eb9-0935f9d44cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:27.920952 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63913516-d81b-4dc9-8eb9-0935f9d44cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:27.921587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63913516-d81b-4dc9-8eb9-0935f9d44cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:27.923527 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63913516-d81b-4dc9-8eb9-0935f9d44cc7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:27.949558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1213/2430] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:38:27 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:38:27.950765 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6f73d-790e-4512-bce3-5893303cc8c8 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:27.961316 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6f73d-790e-4512-bce3-5893303cc8c8 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:27.965917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144034aa-a4db-4682-bbec-83d89c093a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:27.967211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144034aa-a4db-4682-bbec-83d89c093a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:27.967553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144034aa-a4db-4682-bbec-83d89c093a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:27.967840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144034aa-a4db-4682-bbec-83d89c093a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:27.968502 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144034aa-a4db-4682-bbec-83d89c093a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:27.970102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144034aa-a4db-4682-bbec-83d89c093a52 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:27.980273 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1218/2431] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:27.985231 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63913516-d81b-4dc9-8eb9-0935f9d44cc7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:27.995217 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63913516-d81b-4dc9-8eb9-0935f9d44cc7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:28.000008 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054e8c2-9112-4594-b128-6ce51cf56643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:28.000606 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054e8c2-9112-4594-b128-6ce51cf56643 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:28.000919 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054e8c2-9112-4594-b128-6ce51cf56643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:28.001189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054e8c2-9112-4594-b128-6ce51cf56643 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:28.001822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054e8c2-9112-4594-b128-6ce51cf56643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:28.003423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054e8c2-9112-4594-b128-6ce51cf56643 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:28.015841 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1214/2432] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:38:28.020118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144034aa-a4db-4682-bbec-83d89c093a52 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:28.031023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144034aa-a4db-4682-bbec-83d89c093a52 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:28.035639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b8bb3-476e-46e5-8d9d-1b4ff9a5933a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:28.036307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b8bb3-476e-46e5-8d9d-1b4ff9a5933a admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:28.036590 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b8bb3-476e-46e5-8d9d-1b4ff9a5933a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:28.036903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b8bb3-476e-46e5-8d9d-1b4ff9a5933a admin admin] PATH_INFO: `/v3/projects/c437c439c5e84a9fbb4472d15dbd7dd9/users/d2648928319d463987b14386beaeb1a9/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:28.052444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b8bb3-476e-46e5-8d9d-1b4ff9a5933a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c437c439c5e84a9fbb4472d15dbd7dd9, user_id=d2648928319d463987b14386beaeb1a9, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:28.055219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b8bb3-476e-46e5-8d9d-1b4ff9a5933a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:28.068884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1219/2433] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:38:28 2022] PUT /identity/v3/projects/c437c439c5e84a9fbb4472d15dbd7dd9/users/d2648928319d463987b14386beaeb1a9/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:28.135848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaceeae-2927-4daf-9c1a-634f9204d473 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:28.136146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaceeae-2927-4daf-9c1a-634f9204d473 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:28.136418 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaceeae-2927-4daf-9c1a-634f9204d473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:28.165185 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4aaceeae-2927-4daf-9c1a-634f9204d473 None None] MFA Rules not processed for user `d2648928319d463987b14386beaeb1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:28.166416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4aaceeae-2927-4daf-9c1a-634f9204d473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:28.236367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1215/2434] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:28.248026 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b8bb3-476e-46e5-8d9d-1b4ff9a5933a admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:28.249900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-005b8bb3-476e-46e5-8d9d-1b4ff9a5933a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:28.305680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:28.313348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b8bb3-476e-46e5-8d9d-1b4ff9a5933a admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:28.318536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28c7ea-4f5b-47f0-bf2a-c9b3c637bf62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:28.319549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28c7ea-4f5b-47f0-bf2a-c9b3c637bf62 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:28.320065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28c7ea-4f5b-47f0-bf2a-c9b3c637bf62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:28.320402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28c7ea-4f5b-47f0-bf2a-c9b3c637bf62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:28.329997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28c7ea-4f5b-47f0-bf2a-c9b3c637bf62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:28.331511 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28c7ea-4f5b-47f0-bf2a-c9b3c637bf62 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:28.341115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1220/2435] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:38.233507 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:38.336881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:45.827648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbab3a-4520-4b76-98ff-348f9b8e7a3d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:45.828095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbab3a-4520-4b76-98ff-348f9b8e7a3d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:45.828188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbab3a-4520-4b76-98ff-348f9b8e7a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:45.848643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4abbab3a-4520-4b76-98ff-348f9b8e7a3d None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:45.849888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4abbab3a-4520-4b76-98ff-348f9b8e7a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:45.888610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010c0e1-b466-4e52-972c-6a964951aec7 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:45.889170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010c0e1-b466-4e52-972c-6a964951aec7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:45.889264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010c0e1-b466-4e52-972c-6a964951aec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:45.911176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-7010c0e1-b466-4e52-972c-6a964951aec7 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:45.912405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7010c0e1-b466-4e52-972c-6a964951aec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:45.922599 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1216/2436] 10.222.0.55 () {58 vars in 967 bytes} [Fri Aug 19 16:38:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:38:45.971397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbab3a-4520-4b76-98ff-348f9b8e7a3d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:45.972878 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4abbab3a-4520-4b76-98ff-348f9b8e7a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:45.975662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1221/2437] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:38:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:38:45.988278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010c0e1-b466-4e52-972c-6a964951aec7 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:45.991272 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7010c0e1-b466-4e52-972c-6a964951aec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:46.046077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbab3a-4520-4b76-98ff-348f9b8e7a3d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:46.051226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a9417-772d-4e0c-8993-b449602492d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:46.051831 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a9417-772d-4e0c-8993-b449602492d2 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:46.052104 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a9417-772d-4e0c-8993-b449602492d2 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:46.052371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a9417-772d-4e0c-8993-b449602492d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:46.060521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a9417-772d-4e0c-8993-b449602492d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:46.062236 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a9417-772d-4e0c-8993-b449602492d2 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:46.062924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010c0e1-b466-4e52-972c-6a964951aec7 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:46.069375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac2757-ee3f-4346-80f9-1e5bacb7872f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:46.069979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac2757-ee3f-4346-80f9-1e5bacb7872f service swift] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:46.070256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac2757-ee3f-4346-80f9-1e5bacb7872f service swift] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:46.070526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac2757-ee3f-4346-80f9-1e5bacb7872f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:46.070823 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1217/2438] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:38:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 19 16:38:46.079326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac2757-ee3f-4346-80f9-1e5bacb7872f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:46.081005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac2757-ee3f-4346-80f9-1e5bacb7872f service swift] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:46.089472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1222/2439] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:38:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 19 16:38:49.672929 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a9417-772d-4e0c-8993-b449602492d2 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:49.760502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a9417-772d-4e0c-8993-b449602492d2 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:49.770093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:49.770688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:49.770924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:49.771203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] PATH_INFO: `/v3/users/d2648928319d463987b14386beaeb1a9` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:49.772349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2648928319d463987b14386beaeb1a9)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:49.780975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:49.850515 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:49.865703 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2648928319d463987b14386beaeb1a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:49.866488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2648928319d463987b14386beaeb1a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:49.867250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2648928319d463987b14386beaeb1a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:49.895644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2648928319d463987b14386beaeb1a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:49.913189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2648928319d463987b14386beaeb1a9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:49.954135 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1218/2440] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:38:49 2022] DELETE /identity/v3/users/d2648928319d463987b14386beaeb1a9 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:49.958444 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60997a-eb73-413a-9c74-c0d22ee99332 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:49.958941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60997a-eb73-413a-9c74-c0d22ee99332 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:49.959036 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60997a-eb73-413a-9c74-c0d22ee99332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:49.988869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-9d60997a-eb73-413a-9c74-c0d22ee99332 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:49.990628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9d60997a-eb73-413a-9c74-c0d22ee99332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:50.062583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1223/2441] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:50.070439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:50.071920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:50.160549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea5ccc-a5ef-4b74-b957-50bb4a183fdd admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:50.166269 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae87f0-cc28-4b06-9c52-77ca92c49d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:50.167149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae87f0-cc28-4b06-9c52-77ca92c49d43 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:50.167589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae87f0-cc28-4b06-9c52-77ca92c49d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:50.168093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae87f0-cc28-4b06-9c52-77ca92c49d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:50.179361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae87f0-cc28-4b06-9c52-77ca92c49d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:50.181228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae87f0-cc28-4b06-9c52-77ca92c49d43 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:50.194566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1219/2442] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:50.634560 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60997a-eb73-413a-9c74-c0d22ee99332 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:50.640411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9d60997a-eb73-413a-9c74-c0d22ee99332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:50.671530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60997a-eb73-413a-9c74-c0d22ee99332 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:50.677028 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055fac7-723c-40b8-a0fe-eeef44c08426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:50.677874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055fac7-723c-40b8-a0fe-eeef44c08426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:50.678323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055fac7-723c-40b8-a0fe-eeef44c08426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:50.678728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055fac7-723c-40b8-a0fe-eeef44c08426 admin admin] PATH_INFO: `/v3/projects/c437c439c5e84a9fbb4472d15dbd7dd9` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:50.680231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055fac7-723c-40b8-a0fe-eeef44c08426 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c437c439c5e84a9fbb4472d15dbd7dd9)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:50.682263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055fac7-723c-40b8-a0fe-eeef44c08426 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:50.755938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:50.776114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d055fac7-723c-40b8-a0fe-eeef44c08426 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c437c439c5e84a9fbb4472d15dbd7dd9', 'request_id': 'req-d055fac7-723c-40b8-a0fe-eeef44c08426'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:50.809717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d055fac7-723c-40b8-a0fe-eeef44c08426 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c437c439c5e84a9fbb4472d15dbd7dd9', 'request_id': 'req-d055fac7-723c-40b8-a0fe-eeef44c08426'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:50.822050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d055fac7-723c-40b8-a0fe-eeef44c08426 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c437c439c5e84a9fbb4472d15dbd7dd9', 'request_id': 'req-d055fac7-723c-40b8-a0fe-eeef44c08426'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:50.864264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d055fac7-723c-40b8-a0fe-eeef44c08426 admin admin] The token cache is being invalidate because project c437c439c5e84a9fbb4472d15dbd7dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:38:50.865314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d055fac7-723c-40b8-a0fe-eeef44c08426 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:50.865483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d055fac7-723c-40b8-a0fe-eeef44c08426 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:50.869559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1224/2443] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:38:50 2022] DELETE /identity/v3/projects/c437c439c5e84a9fbb4472d15dbd7dd9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:50.880398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d3f39-ffc7-4ca5-ae27-9919a86c4743 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:50.881319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d3f39-ffc7-4ca5-ae27-9919a86c4743 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:50.881670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d3f39-ffc7-4ca5-ae27-9919a86c4743 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:50.884469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1220/2444] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:38:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:38:50.899710 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d5151-6801-4d9f-9c8f-2416238c5b92 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:50.902888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d5151-6801-4d9f-9c8f-2416238c5b92 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:50.903477 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d5151-6801-4d9f-9c8f-2416238c5b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:51.022877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-4a4d5151-6801-4d9f-9c8f-2416238c5b92 None None] MFA Rules not processed for user `1dcbe2b128a94b9abbf3a9b25db1e845`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:38:51.024773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-4a4d5151-6801-4d9f-9c8f-2416238c5b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:51.210330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1225/2445] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:38:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:38:51.224299 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d3f39-ffc7-4ca5-ae27-9919a86c4743 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:51.226298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0f2d3f39-ffc7-4ca5-ae27-9919a86c4743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:51.251034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d3f39-ffc7-4ca5-ae27-9919a86c4743 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:51.256048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ede516-1f47-48c3-a999-da95cebd39dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:51.256670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ede516-1f47-48c3-a999-da95cebd39dd service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:51.257030 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ede516-1f47-48c3-a999-da95cebd39dd service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:51.257313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ede516-1f47-48c3-a999-da95cebd39dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:51.271768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ede516-1f47-48c3-a999-da95cebd39dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:51.273118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ede516-1f47-48c3-a999-da95cebd39dd service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:51.300184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1221/2446] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:38:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:38:55.075474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d5151-6801-4d9f-9c8f-2416238c5b92 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:55.077678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-4a4d5151-6801-4d9f-9c8f-2416238c5b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:55.164133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d5151-6801-4d9f-9c8f-2416238c5b92 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:55.169317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:55.169950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:55.170224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:55.170492 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] PATH_INFO: `/v3/users/cf1371337c03435faa5f1db60168cfbe` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:55.171856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1371337c03435faa5f1db60168cfbe)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:55.175044 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:55.232462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1371337c03435faa5f1db60168cfbe'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.233750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1371337c03435faa5f1db60168cfbe'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.234418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1371337c03435faa5f1db60168cfbe'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.241600 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1371337c03435faa5f1db60168cfbe'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.242937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:55.273305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1371337c03435faa5f1db60168cfbe'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.296483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1226/2447] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:38:55 2022] DELETE /identity/v3/users/cf1371337c03435faa5f1db60168cfbe => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:55.502237 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ede516-1f47-48c3-a999-da95cebd39dd service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:55.516957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ede516-1f47-48c3-a999-da95cebd39dd service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:55.604400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:55.605852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:55.606175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:55.606451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] PATH_INFO: `/v3/projects/a41afd08d1324396940e3acaf49a34b3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:55.607611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a41afd08d1324396940e3acaf49a34b3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:55.609556 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:55.676943 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:38:55.682481 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a41afd08d1324396940e3acaf49a34b3', 'request_id': 'req-9429761d-e71b-44ad-a8f8-8e8717436e71'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.696897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a41afd08d1324396940e3acaf49a34b3', 'request_id': 'req-9429761d-e71b-44ad-a8f8-8e8717436e71'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.707667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a41afd08d1324396940e3acaf49a34b3', 'request_id': 'req-9429761d-e71b-44ad-a8f8-8e8717436e71'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.723430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] The token cache is being invalidate because project a41afd08d1324396940e3acaf49a34b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:38:55.723920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.725187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.726894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1222/2448] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:38:55 2022] DELETE /identity/v3/projects/a41afd08d1324396940e3acaf49a34b3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:38:55.732960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:55.734559 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:55.750751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd838ca3-0c64-410a-9ed5-27fa5992126b admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:55.816558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea873640-5c0e-4cd0-8a84-8cd10d2e436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:55.818104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea873640-5c0e-4cd0-8a84-8cd10d2e436d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:55.819376 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea873640-5c0e-4cd0-8a84-8cd10d2e436d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:55.819609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea873640-5c0e-4cd0-8a84-8cd10d2e436d admin admin] PATH_INFO: `/v3/users/d62cc27342c54b4fb961db960577e748` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:55.823994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea873640-5c0e-4cd0-8a84-8cd10d2e436d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62cc27342c54b4fb961db960577e748)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:55.828389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea873640-5c0e-4cd0-8a84-8cd10d2e436d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:55.903016 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ea873640-5c0e-4cd0-8a84-8cd10d2e436d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62cc27342c54b4fb961db960577e748'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.903691 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ea873640-5c0e-4cd0-8a84-8cd10d2e436d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62cc27342c54b4fb961db960577e748'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.904342 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ea873640-5c0e-4cd0-8a84-8cd10d2e436d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62cc27342c54b4fb961db960577e748'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.912703 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ea873640-5c0e-4cd0-8a84-8cd10d2e436d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62cc27342c54b4fb961db960577e748'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.931538 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ea873640-5c0e-4cd0-8a84-8cd10d2e436d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62cc27342c54b4fb961db960577e748'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:55.960170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1227/2449] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:38:55 2022] DELETE /identity/v3/users/d62cc27342c54b4fb961db960577e748 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:38:56.216423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:38:56.218273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:38:56.263489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429761d-e71b-44ad-a8f8-8e8717436e71 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:38:56.357197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:38:56.357951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:38:56.358233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:38:56.358504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] PATH_INFO: `/v3/projects/d60b3e22e78149a1bb5eab7b3805f48c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:38:56.359637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d60b3e22e78149a1bb5eab7b3805f48c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:38:56.361573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:38:56.424474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd60b3e22e78149a1bb5eab7b3805f48c', 'request_id': 'req-52eac048-6460-484a-928b-08bf33af4e8e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:56.435492 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd60b3e22e78149a1bb5eab7b3805f48c', 'request_id': 'req-52eac048-6460-484a-928b-08bf33af4e8e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:56.441966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd60b3e22e78149a1bb5eab7b3805f48c', 'request_id': 'req-52eac048-6460-484a-928b-08bf33af4e8e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:56.458674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] The token cache is being invalidate because project d60b3e22e78149a1bb5eab7b3805f48c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:38:56.458991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:56.459636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:38:56.461228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1223/2450] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:38:56 2022] DELETE /identity/v3/projects/d60b3e22e78149a1bb5eab7b3805f48c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:39:03.754162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097e2ec-405b-47f4-89de-c94cddd28064 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:03.754645 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097e2ec-405b-47f4-89de-c94cddd28064 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:03.754757 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097e2ec-405b-47f4-89de-c94cddd28064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:03.775417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-7097e2ec-405b-47f4-89de-c94cddd28064 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:03.776612 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7097e2ec-405b-47f4-89de-c94cddd28064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:03.822887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:03.832960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1228/2451] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:39:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:39:03.845847 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:03.847339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:03.916684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eac048-6460-484a-928b-08bf33af4e8e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:03.922042 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a24245-a1b3-4028-a082-830c0592d8ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:03.922677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a24245-a1b3-4028-a082-830c0592d8ab service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:03.922961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a24245-a1b3-4028-a082-830c0592d8ab service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:03.923239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a24245-a1b3-4028-a082-830c0592d8ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:03.928200 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:03.933007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a24245-a1b3-4028-a082-830c0592d8ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:03.935589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a24245-a1b3-4028-a082-830c0592d8ab service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:03.944759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1224/2452] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:39:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:39:13.833944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:13.941613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:20.367224 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85700f77-0655-4d63-bf77-201b2af39500 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:20.367702 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85700f77-0655-4d63-bf77-201b2af39500 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:20.367797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85700f77-0655-4d63-bf77-201b2af39500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:20.388314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-85700f77-0655-4d63-bf77-201b2af39500 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:20.389581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-85700f77-0655-4d63-bf77-201b2af39500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:20.396815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1229/2453] 10.222.0.55 () {58 vars in 967 bytes} [Fri Aug 19 16:39:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:39:20.409007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a24245-a1b3-4028-a082-830c0592d8ab service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:20.419817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a24245-a1b3-4028-a082-830c0592d8ab service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:20.424888 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1604c-2689-4be9-9b8b-f85ae54c3305 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:20.425482 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1604c-2689-4be9-9b8b-f85ae54c3305 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:20.425762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1604c-2689-4be9-9b8b-f85ae54c3305 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:20.426036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1604c-2689-4be9-9b8b-f85ae54c3305 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:20.434798 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1604c-2689-4be9-9b8b-f85ae54c3305 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:20.436378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1604c-2689-4be9-9b8b-f85ae54c3305 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:20.444592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1225/2454] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:39:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:39:30.382109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:30.441194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=6 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:48.325207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85700f77-0655-4d63-bf77-201b2af39500 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:48.326849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-85700f77-0655-4d63-bf77-201b2af39500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:48.420067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85700f77-0655-4d63-bf77-201b2af39500 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:48.424931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b2dff-913f-49b3-b585-e28639212bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:48.425550 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b2dff-913f-49b3-b585-e28639212bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:48.425827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b2dff-913f-49b3-b585-e28639212bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:48.426251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b2dff-913f-49b3-b585-e28639212bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:48.428242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5f2b2dff-913f-49b3-b585-e28639212bb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:48.560409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b2dff-913f-49b3-b585-e28639212bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:48.562013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b2dff-913f-49b3-b585-e28639212bb1 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:48.591870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1230/2455] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:39:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:39:51.813453 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1604c-2689-4be9-9b8b-f85ae54c3305 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:51.815647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f8f1604c-2689-4be9-9b8b-f85ae54c3305 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:51.888017 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b2dff-913f-49b3-b585-e28639212bb1 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:51.890759 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5f2b2dff-913f-49b3-b585-e28639212bb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:51.907161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b2dff-913f-49b3-b585-e28639212bb1 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:51.912143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3922cf-a987-4964-abb9-23ae6d79e6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:51.912808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3922cf-a987-4964-abb9-23ae6d79e6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:51.913091 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3922cf-a987-4964-abb9-23ae6d79e6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:51.913363 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3922cf-a987-4964-abb9-23ae6d79e6bc admin admin] PATH_INFO: `/v3/users/9b0455ae3ec94c929c940df6a0f6687d` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:51.915202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3922cf-a987-4964-abb9-23ae6d79e6bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0455ae3ec94c929c940df6a0f6687d)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:51.917330 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3922cf-a987-4964-abb9-23ae6d79e6bc admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:51.948019 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:51.965816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1604c-2689-4be9-9b8b-f85ae54c3305 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:51.967764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6f3922cf-a987-4964-abb9-23ae6d79e6bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0455ae3ec94c929c940df6a0f6687d'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:51.968413 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6f3922cf-a987-4964-abb9-23ae6d79e6bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0455ae3ec94c929c940df6a0f6687d'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:51.969048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6f3922cf-a987-4964-abb9-23ae6d79e6bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0455ae3ec94c929c940df6a0f6687d'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:51.971110 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d85efd-483f-43b5-88de-b8b24da427e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:51.971720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d85efd-483f-43b5-88de-b8b24da427e2 service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:51.972024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d85efd-483f-43b5-88de-b8b24da427e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:51.972296 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d85efd-483f-43b5-88de-b8b24da427e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:51.974055 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-81d85efd-483f-43b5-88de-b8b24da427e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:51.976240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6f3922cf-a987-4964-abb9-23ae6d79e6bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0455ae3ec94c929c940df6a0f6687d'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:51.986683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d85efd-483f-43b5-88de-b8b24da427e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:51.988430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d85efd-483f-43b5-88de-b8b24da427e2 service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:51.989859 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6f3922cf-a987-4964-abb9-23ae6d79e6bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0455ae3ec94c929c940df6a0f6687d'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:51.999787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1226/2456] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:39:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:39:52.003168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1231/2457] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:39:51 2022] DELETE /identity/v3/users/9b0455ae3ec94c929c940df6a0f6687d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:39:52.188264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d85efd-483f-43b5-88de-b8b24da427e2 service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:52.190084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-81d85efd-483f-43b5-88de-b8b24da427e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:52.205117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d85efd-483f-43b5-88de-b8b24da427e2 service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:52.263126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18921465-9fab-4b3f-a53e-adced07655a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:52.263787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18921465-9fab-4b3f-a53e-adced07655a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:52.264069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18921465-9fab-4b3f-a53e-adced07655a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:52.264375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18921465-9fab-4b3f-a53e-adced07655a6 admin admin] PATH_INFO: `/v3/projects/1743538991c8415394ac6cd5aed28d6a` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:52.265533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18921465-9fab-4b3f-a53e-adced07655a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1743538991c8415394ac6cd5aed28d6a)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:52.267454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18921465-9fab-4b3f-a53e-adced07655a6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:52.308534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:52.316190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-18921465-9fab-4b3f-a53e-adced07655a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1743538991c8415394ac6cd5aed28d6a', 'request_id': 'req-18921465-9fab-4b3f-a53e-adced07655a6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:52.322528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-18921465-9fab-4b3f-a53e-adced07655a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1743538991c8415394ac6cd5aed28d6a', 'request_id': 'req-18921465-9fab-4b3f-a53e-adced07655a6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:52.328550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-18921465-9fab-4b3f-a53e-adced07655a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1743538991c8415394ac6cd5aed28d6a', 'request_id': 'req-18921465-9fab-4b3f-a53e-adced07655a6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:52.340696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-18921465-9fab-4b3f-a53e-adced07655a6 admin admin] The token cache is being invalidate because project 1743538991c8415394ac6cd5aed28d6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:39:52.341007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-18921465-9fab-4b3f-a53e-adced07655a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:52.341643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-18921465-9fab-4b3f-a53e-adced07655a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:52.343107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1227/2458] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:39:52 2022] DELETE /identity/v3/projects/1743538991c8415394ac6cd5aed28d6a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:39:52.348140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1425db0-a55b-4de7-b2a9-d1d20ea945a2 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:52.348426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1425db0-a55b-4de7-b2a9-d1d20ea945a2 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:52.348701 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1425db0-a55b-4de7-b2a9-d1d20ea945a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:52.405687 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b1425db0-a55b-4de7-b2a9-d1d20ea945a2 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:52.406816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b1425db0-a55b-4de7-b2a9-d1d20ea945a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:52.464482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1232/2459] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:39:52.472451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48239f9-769c-4786-8ec3-237c12270334 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:52.472782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48239f9-769c-4786-8ec3-237c12270334 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:52.473071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48239f9-769c-4786-8ec3-237c12270334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:52.492516 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a48239f9-769c-4786-8ec3-237c12270334 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:52.493778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a48239f9-769c-4786-8ec3-237c12270334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:52.501631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1228/2460] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:39:52.505812 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1425db0-a55b-4de7-b2a9-d1d20ea945a2 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:52.516115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1425db0-a55b-4de7-b2a9-d1d20ea945a2 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:52.520884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b125b2b-5a7e-4d56-a897-d060a63e2c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:52.521472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b125b2b-5a7e-4d56-a897-d060a63e2c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:52.521746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b125b2b-5a7e-4d56-a897-d060a63e2c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:52.522011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b125b2b-5a7e-4d56-a897-d060a63e2c36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:52.522639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b125b2b-5a7e-4d56-a897-d060a63e2c36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:39:52.522933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b125b2b-5a7e-4d56-a897-d060a63e2c36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:52.524580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b125b2b-5a7e-4d56-a897-d060a63e2c36 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:52.536796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1233/2461] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:39:52.540978 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48239f9-769c-4786-8ec3-237c12270334 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:52.550155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48239f9-769c-4786-8ec3-237c12270334 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:52.554704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a01869-3762-4720-b003-2f21355e4a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:52.555298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a01869-3762-4720-b003-2f21355e4a8b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:52.555598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a01869-3762-4720-b003-2f21355e4a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:52.555869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a01869-3762-4720-b003-2f21355e4a8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:52.556479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a01869-3762-4720-b003-2f21355e4a8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:52.558477 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a01869-3762-4720-b003-2f21355e4a8b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:52.586601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1229/2462] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:39:52 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:39:52.590898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b125b2b-5a7e-4d56-a897-d060a63e2c36 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:52.600944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b125b2b-5a7e-4d56-a897-d060a63e2c36 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:52.657749 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53cf92-527c-4f84-8b86-ad54cbd3c556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:52.657978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53cf92-527c-4f84-8b86-ad54cbd3c556 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:52.658254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53cf92-527c-4f84-8b86-ad54cbd3c556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:52.658524 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53cf92-527c-4f84-8b86-ad54cbd3c556 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:52.659153 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53cf92-527c-4f84-8b86-ad54cbd3c556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:52.661140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53cf92-527c-4f84-8b86-ad54cbd3c556 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:52.669929 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:52.683774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1234/2463] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:39:52 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:39:52.687886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a01869-3762-4720-b003-2f21355e4a8b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:52.699167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a01869-3762-4720-b003-2f21355e4a8b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:52.703947 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fe67b-ff78-499a-9bd2-6932267735a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:52.704551 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fe67b-ff78-499a-9bd2-6932267735a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:52.704889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fe67b-ff78-499a-9bd2-6932267735a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:52.705167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fe67b-ff78-499a-9bd2-6932267735a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:52.705811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fe67b-ff78-499a-9bd2-6932267735a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:52.707466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fe67b-ff78-499a-9bd2-6932267735a6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:52.717428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1230/2464] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:39:52.721447 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53cf92-527c-4f84-8b86-ad54cbd3c556 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:52.731454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53cf92-527c-4f84-8b86-ad54cbd3c556 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:52.735988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94edca-8672-43a3-b876-b6b84c30dd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:52.736576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94edca-8672-43a3-b876-b6b84c30dd3f admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:52.736888 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94edca-8672-43a3-b876-b6b84c30dd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:52.737161 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94edca-8672-43a3-b876-b6b84c30dd3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:52.737791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94edca-8672-43a3-b876-b6b84c30dd3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:52.739403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94edca-8672-43a3-b876-b6b84c30dd3f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:52.750301 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1235/2465] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:39:52.754519 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fe67b-ff78-499a-9bd2-6932267735a6 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:52.765231 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fe67b-ff78-499a-9bd2-6932267735a6 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:52.769925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da5bf1-ea63-4b93-8134-4505ff1cf11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:52.770566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da5bf1-ea63-4b93-8134-4505ff1cf11b admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:52.770850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da5bf1-ea63-4b93-8134-4505ff1cf11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:52.771126 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da5bf1-ea63-4b93-8134-4505ff1cf11b admin admin] PATH_INFO: `/v3/projects/25c65e9c4d3d4e0f83d3996adfa9e9ea/users/78b5db5d32e0479c98f77b8119a02eb5/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:52.785591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da5bf1-ea63-4b93-8134-4505ff1cf11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c65e9c4d3d4e0f83d3996adfa9e9ea, user_id=78b5db5d32e0479c98f77b8119a02eb5, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:52.788398 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da5bf1-ea63-4b93-8134-4505ff1cf11b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:52.802744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1231/2466] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:39:52 2022] PUT /identity/v3/projects/25c65e9c4d3d4e0f83d3996adfa9e9ea/users/78b5db5d32e0479c98f77b8119a02eb5/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:39:52.814931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a612a5a-897b-465d-9fa8-b07b764f8d81 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:52.815217 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a612a5a-897b-465d-9fa8-b07b764f8d81 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:52.815489 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a612a5a-897b-465d-9fa8-b07b764f8d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:52.843388 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-6a612a5a-897b-465d-9fa8-b07b764f8d81 None None] MFA Rules not processed for user `78b5db5d32e0479c98f77b8119a02eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:52.844603 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6a612a5a-897b-465d-9fa8-b07b764f8d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:52.913054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1236/2467] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:39:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:39:52.920889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da5bf1-ea63-4b93-8134-4505ff1cf11b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:52.922651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-34da5bf1-ea63-4b93-8134-4505ff1cf11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:53.037762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da5bf1-ea63-4b93-8134-4505ff1cf11b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:53.044530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449529b-d80b-412b-9105-90e6043791f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:53.045178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449529b-d80b-412b-9105-90e6043791f0 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:53.045457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449529b-d80b-412b-9105-90e6043791f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:53.046080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449529b-d80b-412b-9105-90e6043791f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:53.054211 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449529b-d80b-412b-9105-90e6043791f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:53.056172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449529b-d80b-412b-9105-90e6043791f0 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:53.066178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1232/2468] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:39:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:39:54.230882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a612a5a-897b-465d-9fa8-b07b764f8d81 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:54.232291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6a612a5a-897b-465d-9fa8-b07b764f8d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:54.305570 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a612a5a-897b-465d-9fa8-b07b764f8d81 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:54.309366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa240f3-541b-4774-9bc3-bd4ef2a8f4fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:54.310052 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa240f3-541b-4774-9bc3-bd4ef2a8f4fc service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:54.310800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa240f3-541b-4774-9bc3-bd4ef2a8f4fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:54.310800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa240f3-541b-4774-9bc3-bd4ef2a8f4fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:54.312779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-efa240f3-541b-4774-9bc3-bd4ef2a8f4fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:54.391556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa240f3-541b-4774-9bc3-bd4ef2a8f4fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:54.392938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa240f3-541b-4774-9bc3-bd4ef2a8f4fc service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:54.405262 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1237/2469] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:39:55.810475 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449529b-d80b-412b-9105-90e6043791f0 service nova] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:55.819322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449529b-d80b-412b-9105-90e6043791f0 service nova] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:55.824922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:55.825651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:55.825932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:55.826208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] PATH_INFO: `/v3/users/78b5db5d32e0479c98f77b8119a02eb5` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:55.827372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b5db5d32e0479c98f77b8119a02eb5)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:55.829348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:55.854145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:55.880669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b5db5d32e0479c98f77b8119a02eb5'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:55.881379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b5db5d32e0479c98f77b8119a02eb5'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:55.882216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b5db5d32e0479c98f77b8119a02eb5'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:55.894957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b5db5d32e0479c98f77b8119a02eb5'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:55.902923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b5db5d32e0479c98f77b8119a02eb5'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:55.917285 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1233/2470] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:39:55 2022] DELETE /identity/v3/users/78b5db5d32e0479c98f77b8119a02eb5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:39:55.921718 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca5b24-1f0e-4447-822b-67d3d808d530 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:55.922201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca5b24-1f0e-4447-822b-67d3d808d530 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:55.922317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca5b24-1f0e-4447-822b-67d3d808d530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:55.943588 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d0ca5b24-1f0e-4447-822b-67d3d808d530 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:55.944843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d0ca5b24-1f0e-4447-822b-67d3d808d530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:55.965573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:56.029336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1238/2471] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:39:56.041213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:56.043897 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:56.130788 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d5d31-d4e8-483a-a98b-077ef47b856a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:56.134893 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c66499-9dac-4dec-918e-af7670baa961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:56.135518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c66499-9dac-4dec-918e-af7670baa961 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:56.135803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c66499-9dac-4dec-918e-af7670baa961 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:56.136162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c66499-9dac-4dec-918e-af7670baa961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:56.150085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c66499-9dac-4dec-918e-af7670baa961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:56.152238 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c66499-9dac-4dec-918e-af7670baa961 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:56.166800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1234/2472] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:39:56.548435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca5b24-1f0e-4447-822b-67d3d808d530 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:56.549960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d0ca5b24-1f0e-4447-822b-67d3d808d530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:56.570826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca5b24-1f0e-4447-822b-67d3d808d530 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:56.576737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d866de-912b-49f3-97d0-f44a5a9e9318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:56.577438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d866de-912b-49f3-97d0-f44a5a9e9318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:56.577721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d866de-912b-49f3-97d0-f44a5a9e9318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:56.577998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d866de-912b-49f3-97d0-f44a5a9e9318 admin admin] PATH_INFO: `/v3/projects/25c65e9c4d3d4e0f83d3996adfa9e9ea` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:56.580363 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d866de-912b-49f3-97d0-f44a5a9e9318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25c65e9c4d3d4e0f83d3996adfa9e9ea)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:56.582361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d866de-912b-49f3-97d0-f44a5a9e9318 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:56.663023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b6d866de-912b-49f3-97d0-f44a5a9e9318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25c65e9c4d3d4e0f83d3996adfa9e9ea', 'request_id': 'req-b6d866de-912b-49f3-97d0-f44a5a9e9318'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:56.672022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b6d866de-912b-49f3-97d0-f44a5a9e9318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25c65e9c4d3d4e0f83d3996adfa9e9ea', 'request_id': 'req-b6d866de-912b-49f3-97d0-f44a5a9e9318'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:56.678361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b6d866de-912b-49f3-97d0-f44a5a9e9318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25c65e9c4d3d4e0f83d3996adfa9e9ea', 'request_id': 'req-b6d866de-912b-49f3-97d0-f44a5a9e9318'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:56.696495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b6d866de-912b-49f3-97d0-f44a5a9e9318 admin admin] The token cache is being invalidate because project 25c65e9c4d3d4e0f83d3996adfa9e9ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:39:56.696861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b6d866de-912b-49f3-97d0-f44a5a9e9318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:56.697516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b6d866de-912b-49f3-97d0-f44a5a9e9318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:56.699973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1239/2473] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:39:56 2022] DELETE /identity/v3/projects/25c65e9c4d3d4e0f83d3996adfa9e9ea => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:39:56.705568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e71982-324f-4263-af8b-fa8009890557 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:56.705860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e71982-324f-4263-af8b-fa8009890557 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:56.706141 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e71982-324f-4263-af8b-fa8009890557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:56.729625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-39e71982-324f-4263-af8b-fa8009890557 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:56.731065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-39e71982-324f-4263-af8b-fa8009890557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:56.803636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1235/2474] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:39:56.812474 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d081c-0430-45ce-908b-816d48b45fe5 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:56.812772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d081c-0430-45ce-908b-816d48b45fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:56.813080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d081c-0430-45ce-908b-816d48b45fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:56.834846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-796d081c-0430-45ce-908b-816d48b45fe5 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:56.836584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-796d081c-0430-45ce-908b-816d48b45fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:56.846823 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1240/2475] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:39:56.850959 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e71982-324f-4263-af8b-fa8009890557 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:56.862147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e71982-324f-4263-af8b-fa8009890557 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:56.868371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c3127-675a-48c5-8488-a653771a18dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:56.869270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c3127-675a-48c5-8488-a653771a18dd admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:56.869816 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c3127-675a-48c5-8488-a653771a18dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:56.869930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c3127-675a-48c5-8488-a653771a18dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:56.870616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c3127-675a-48c5-8488-a653771a18dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:39:56.871011 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c3127-675a-48c5-8488-a653771a18dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:56.872721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c3127-675a-48c5-8488-a653771a18dd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:56.901233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1236/2476] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:39:56.902543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d081c-0430-45ce-908b-816d48b45fe5 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:56.918619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d081c-0430-45ce-908b-816d48b45fe5 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:56.923737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f76de-f62c-46af-86a1-7232d96b14ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:56.924782 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f76de-f62c-46af-86a1-7232d96b14ee admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:56.925004 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f76de-f62c-46af-86a1-7232d96b14ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:56.925274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f76de-f62c-46af-86a1-7232d96b14ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:56.925904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f76de-f62c-46af-86a1-7232d96b14ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:56.927900 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f76de-f62c-46af-86a1-7232d96b14ee admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:56.968870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1241/2477] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:39:56 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:39:56.971916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c3127-675a-48c5-8488-a653771a18dd admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:56.984284 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c3127-675a-48c5-8488-a653771a18dd admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:57.012530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:57.048234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf84f01-b2e7-4f41-9b7f-78f3da64de2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:57.048905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf84f01-b2e7-4f41-9b7f-78f3da64de2b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:57.049186 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf84f01-b2e7-4f41-9b7f-78f3da64de2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:57.049459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf84f01-b2e7-4f41-9b7f-78f3da64de2b admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:57.050108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf84f01-b2e7-4f41-9b7f-78f3da64de2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:57.052039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf84f01-b2e7-4f41-9b7f-78f3da64de2b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:57.078795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1237/2478] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:39:56 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:39:57.079963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f76de-f62c-46af-86a1-7232d96b14ee admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:57.090973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f76de-f62c-46af-86a1-7232d96b14ee admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:57.096096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b21cc-b08c-4d54-8785-b9370688c75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:57.096709 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b21cc-b08c-4d54-8785-b9370688c75a admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:57.097021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b21cc-b08c-4d54-8785-b9370688c75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:57.097294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b21cc-b08c-4d54-8785-b9370688c75a admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:57.097930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b21cc-b08c-4d54-8785-b9370688c75a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:57.099556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b21cc-b08c-4d54-8785-b9370688c75a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:57.114990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1242/2479] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:39:57.115554 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf84f01-b2e7-4f41-9b7f-78f3da64de2b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:57.126653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf84f01-b2e7-4f41-9b7f-78f3da64de2b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:57.132029 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834d367-5667-484e-ae5e-dae833698282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:57.132652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834d367-5667-484e-ae5e-dae833698282 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:57.132962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834d367-5667-484e-ae5e-dae833698282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:57.133237 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834d367-5667-484e-ae5e-dae833698282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:57.134088 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834d367-5667-484e-ae5e-dae833698282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:57.135696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834d367-5667-484e-ae5e-dae833698282 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:57.147327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1238/2480] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:39:57.152108 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b21cc-b08c-4d54-8785-b9370688c75a admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:57.162806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b21cc-b08c-4d54-8785-b9370688c75a admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:57.168969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bb519-f4b9-4e09-840e-5101efee7444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:57.169786 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bb519-f4b9-4e09-840e-5101efee7444 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:57.170067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bb519-f4b9-4e09-840e-5101efee7444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:57.170356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bb519-f4b9-4e09-840e-5101efee7444 admin admin] PATH_INFO: `/v3/projects/8bd3a3c6fa9044868cee42d4e57872d1/users/8e2a731ac9a840479d87e173b0dd5328/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:57.187245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bb519-f4b9-4e09-840e-5101efee7444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd3a3c6fa9044868cee42d4e57872d1, user_id=8e2a731ac9a840479d87e173b0dd5328, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:57.190662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bb519-f4b9-4e09-840e-5101efee7444 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:57.206410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1243/2481] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:39:57 2022] PUT /identity/v3/projects/8bd3a3c6fa9044868cee42d4e57872d1/users/8e2a731ac9a840479d87e173b0dd5328/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:39:57.218049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4645ad-a33d-408f-abb7-a044acd6984c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:57.218411 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4645ad-a33d-408f-abb7-a044acd6984c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:57.218668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4645ad-a33d-408f-abb7-a044acd6984c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:57.274858 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9d4645ad-a33d-408f-abb7-a044acd6984c None None] MFA Rules not processed for user `8e2a731ac9a840479d87e173b0dd5328`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:57.277198 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9d4645ad-a33d-408f-abb7-a044acd6984c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:57.378115 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1239/2482] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:39:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:39:57.379741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bb519-f4b9-4e09-840e-5101efee7444 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:57.437356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:57.459523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bb519-f4b9-4e09-840e-5101efee7444 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:57.467420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9970a2d-740f-42d3-b0fe-24bf907be783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:57.467894 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9970a2d-740f-42d3-b0fe-24bf907be783 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:57.468241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9970a2d-740f-42d3-b0fe-24bf907be783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:57.468584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9970a2d-740f-42d3-b0fe-24bf907be783 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:57.469335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9970a2d-740f-42d3-b0fe-24bf907be783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:57.471382 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9970a2d-740f-42d3-b0fe-24bf907be783 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:57.499902 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1244/2483] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:39:57 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:39:57.503774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4645ad-a33d-408f-abb7-a044acd6984c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:57.515213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4645ad-a33d-408f-abb7-a044acd6984c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:57.573288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfa039-c274-478f-a046-df1823801846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:57.573906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfa039-c274-478f-a046-df1823801846 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:57.574209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfa039-c274-478f-a046-df1823801846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:57.574482 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfa039-c274-478f-a046-df1823801846 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:57.575082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfa039-c274-478f-a046-df1823801846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:57.577023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfa039-c274-478f-a046-df1823801846 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:57.601413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1240/2484] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:39:57 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:39:57.602819 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9970a2d-740f-42d3-b0fe-24bf907be783 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:57.618577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9970a2d-740f-42d3-b0fe-24bf907be783 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:57.623754 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae84c9-2ec7-4d90-97c5-fedbe9bdbc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:57.624401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae84c9-2ec7-4d90-97c5-fedbe9bdbc90 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:57.624680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae84c9-2ec7-4d90-97c5-fedbe9bdbc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:57.624982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae84c9-2ec7-4d90-97c5-fedbe9bdbc90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:57.626297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae84c9-2ec7-4d90-97c5-fedbe9bdbc90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:57.627904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae84c9-2ec7-4d90-97c5-fedbe9bdbc90 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:57.641226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1245/2485] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:39:57.645075 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfa039-c274-478f-a046-df1823801846 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:57.659757 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfa039-c274-478f-a046-df1823801846 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:57.665348 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc4a45-c3b5-434e-b515-2bca46207afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:57.668790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc4a45-c3b5-434e-b515-2bca46207afe admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:57.668790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc4a45-c3b5-434e-b515-2bca46207afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:57.668790 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc4a45-c3b5-434e-b515-2bca46207afe admin admin] PATH_INFO: `/v3/projects/2844cf567c3340d1af281c7910cb590f/users/a3df3461859a42e2a2e132bb1ad64e77/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:57.683012 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc4a45-c3b5-434e-b515-2bca46207afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2844cf567c3340d1af281c7910cb590f, user_id=a3df3461859a42e2a2e132bb1ad64e77, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:57.685791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc4a45-c3b5-434e-b515-2bca46207afe admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:57.699976 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:57.708971 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1241/2486] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:39:57 2022] PUT /identity/v3/projects/2844cf567c3340d1af281c7910cb590f/users/a3df3461859a42e2a2e132bb1ad64e77/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:39:57.711919 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae84c9-2ec7-4d90-97c5-fedbe9bdbc90 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:57.724231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae84c9-2ec7-4d90-97c5-fedbe9bdbc90 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:57.875580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4ae81-efe0-4734-9788-6a16a5a5f42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:57.876461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4ae81-efe0-4734-9788-6a16a5a5f42f admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:57.877024 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4ae81-efe0-4734-9788-6a16a5a5f42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:57.877517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4ae81-efe0-4734-9788-6a16a5a5f42f admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:57.878387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4ae81-efe0-4734-9788-6a16a5a5f42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:57.880242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4ae81-efe0-4734-9788-6a16a5a5f42f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:57.909099 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1246/2487] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:39:57.911372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc4a45-c3b5-434e-b515-2bca46207afe admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:57.927458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc4a45-c3b5-434e-b515-2bca46207afe admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:57.935848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154bf84-c77c-452a-8e83-c5e0d8c95e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:57.937431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154bf84-c77c-452a-8e83-c5e0d8c95e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:57.937942 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154bf84-c77c-452a-8e83-c5e0d8c95e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:57.938450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154bf84-c77c-452a-8e83-c5e0d8c95e82 admin admin] PATH_INFO: `/v3/projects/2844cf567c3340d1af281c7910cb590f/users/a3df3461859a42e2a2e132bb1ad64e77/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:57.941070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154bf84-c77c-452a-8e83-c5e0d8c95e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2844cf567c3340d1af281c7910cb590f, user_id=a3df3461859a42e2a2e132bb1ad64e77, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:57.944784 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154bf84-c77c-452a-8e83-c5e0d8c95e82 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:57.966434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1242/2488] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:39:57 2022] PUT /identity/v3/projects/2844cf567c3340d1af281c7910cb590f/users/a3df3461859a42e2a2e132bb1ad64e77/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:39:57.979266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33d274-0033-4e52-9575-774f308e47e2 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:57.979794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33d274-0033-4e52-9575-774f308e47e2 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:57.979887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33d274-0033-4e52-9575-774f308e47e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:58.023222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-bb33d274-0033-4e52-9575-774f308e47e2 None None] MFA Rules not processed for user `a3df3461859a42e2a2e132bb1ad64e77`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:58.026190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-bb33d274-0033-4e52-9575-774f308e47e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:58.138544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1247/2489] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:39:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:39:58.153560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154bf84-c77c-452a-8e83-c5e0d8c95e82 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:58.156110 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e154bf84-c77c-452a-8e83-c5e0d8c95e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:58.253164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154bf84-c77c-452a-8e83-c5e0d8c95e82 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:58.257028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2d20-20d7-45ad-9c38-56678ba6237b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:58.258648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2d20-20d7-45ad-9c38-56678ba6237b service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:58.258825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2d20-20d7-45ad-9c38-56678ba6237b service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:58.259086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2d20-20d7-45ad-9c38-56678ba6237b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:58.270161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa2d20-20d7-45ad-9c38-56678ba6237b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:58.271966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa2d20-20d7-45ad-9c38-56678ba6237b service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:58.288102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1243/2490] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:39:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:39:58.853776 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33d274-0033-4e52-9575-774f308e47e2 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:58.855817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-bb33d274-0033-4e52-9575-774f308e47e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:58.916111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:58.952975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33d274-0033-4e52-9575-774f308e47e2 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:58.958774 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:58.959449 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:58.959803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:58.960151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] PATH_INFO: `/v3/users/d55dc52ab80642da90f05e74092ffdf8` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:58.962098 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d55dc52ab80642da90f05e74092ffdf8)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:58.964261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:59.048743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd55dc52ab80642da90f05e74092ffdf8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.049607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd55dc52ab80642da90f05e74092ffdf8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.051608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd55dc52ab80642da90f05e74092ffdf8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.067985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd55dc52ab80642da90f05e74092ffdf8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.090095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd55dc52ab80642da90f05e74092ffdf8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.123439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1248/2491] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:39:58 2022] DELETE /identity/v3/users/d55dc52ab80642da90f05e74092ffdf8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:39:59.195213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2d20-20d7-45ad-9c38-56678ba6237b service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:59.208499 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2d20-20d7-45ad-9c38-56678ba6237b service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:59.301020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85df5d-8fad-483d-bba9-29390393dd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:59.301473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85df5d-8fad-483d-bba9-29390393dd34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:59.301763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85df5d-8fad-483d-bba9-29390393dd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:59.302848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85df5d-8fad-483d-bba9-29390393dd34 admin admin] PATH_INFO: `/v3/users/8e2a731ac9a840479d87e173b0dd5328` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:59.305408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85df5d-8fad-483d-bba9-29390393dd34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e2a731ac9a840479d87e173b0dd5328)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:59.307759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85df5d-8fad-483d-bba9-29390393dd34 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:59.376384 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3f85df5d-8fad-483d-bba9-29390393dd34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e2a731ac9a840479d87e173b0dd5328'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.377774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3f85df5d-8fad-483d-bba9-29390393dd34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e2a731ac9a840479d87e173b0dd5328'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.378618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3f85df5d-8fad-483d-bba9-29390393dd34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e2a731ac9a840479d87e173b0dd5328'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.385773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:59.387465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:59.403760 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3f85df5d-8fad-483d-bba9-29390393dd34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e2a731ac9a840479d87e173b0dd5328'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.408872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4288c-99af-4365-8606-fa816aef386f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:59.419653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3f85df5d-8fad-483d-bba9-29390393dd34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e2a731ac9a840479d87e173b0dd5328'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.424822 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a097c9d-4255-4d81-86fc-0e442c955dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:59.426144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a097c9d-4255-4d81-86fc-0e442c955dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:59.426908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a097c9d-4255-4d81-86fc-0e442c955dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:59.427879 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a097c9d-4255-4d81-86fc-0e442c955dd5 admin admin] PATH_INFO: `/v3/projects/e15ca27a2c874997bd01de973a948311` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:59.429314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a097c9d-4255-4d81-86fc-0e442c955dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e15ca27a2c874997bd01de973a948311)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:59.431649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a097c9d-4255-4d81-86fc-0e442c955dd5 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:59.448979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1244/2492] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:39:59 2022] DELETE /identity/v3/users/8e2a731ac9a840479d87e173b0dd5328 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:39:59.453101 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d087550-778c-4515-8cff-1f64fff88573 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:59.453473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d087550-778c-4515-8cff-1f64fff88573 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:59.453801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d087550-778c-4515-8cff-1f64fff88573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:59.475067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4d087550-778c-4515-8cff-1f64fff88573 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:59.476881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4d087550-778c-4515-8cff-1f64fff88573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:59.481633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:59.503886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6a097c9d-4255-4d81-86fc-0e442c955dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e15ca27a2c874997bd01de973a948311', 'request_id': 'req-6a097c9d-4255-4d81-86fc-0e442c955dd5'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.510094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6a097c9d-4255-4d81-86fc-0e442c955dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e15ca27a2c874997bd01de973a948311', 'request_id': 'req-6a097c9d-4255-4d81-86fc-0e442c955dd5'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.516163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6a097c9d-4255-4d81-86fc-0e442c955dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e15ca27a2c874997bd01de973a948311', 'request_id': 'req-6a097c9d-4255-4d81-86fc-0e442c955dd5'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.530660 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6a097c9d-4255-4d81-86fc-0e442c955dd5 admin admin] The token cache is being invalidate because project e15ca27a2c874997bd01de973a948311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:39:59.530955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6a097c9d-4255-4d81-86fc-0e442c955dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.531581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-6a097c9d-4255-4d81-86fc-0e442c955dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:39:59.533315 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1249/2493] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:39:59 2022] DELETE /identity/v3/projects/e15ca27a2c874997bd01de973a948311 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:39:59.537854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad821b9-68d3-49d2-ba0a-40c38615698e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:59.538149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad821b9-68d3-49d2-ba0a-40c38615698e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:59.538432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad821b9-68d3-49d2-ba0a-40c38615698e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:59.544473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1245/2494] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:39:59.554102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d087550-778c-4515-8cff-1f64fff88573 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:59.555845 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4d087550-778c-4515-8cff-1f64fff88573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:59.558836 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-0ad821b9-68d3-49d2-ba0a-40c38615698e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:59.560057 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0ad821b9-68d3-49d2-ba0a-40c38615698e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:59.595450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:39:59.618894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d087550-778c-4515-8cff-1f64fff88573 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:59.628208 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1250/2495] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:39:59.637177 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38f96a-08f3-48c2-8da0-ec135a6665c9 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:59.637472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38f96a-08f3-48c2-8da0-ec135a6665c9 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:59.637748 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38f96a-08f3-48c2-8da0-ec135a6665c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:59.658587 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-6b38f96a-08f3-48c2-8da0-ec135a6665c9 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:39:59.659419 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6b38f96a-08f3-48c2-8da0-ec135a6665c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:59.669928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1251/2496] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:39:59.674604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38f96a-08f3-48c2-8da0-ec135a6665c9 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:59.681583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1979a2f-309f-4e7c-be69-44a8f9152fae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:59.682271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1979a2f-309f-4e7c-be69-44a8f9152fae service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:59.682586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1979a2f-309f-4e7c-be69-44a8f9152fae service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:59.682892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1979a2f-309f-4e7c-be69-44a8f9152fae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:59.684798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38f96a-08f3-48c2-8da0-ec135a6665c9 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:59.689813 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32678a83-4ef6-43d8-972a-34cb1038b5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:59.690399 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32678a83-4ef6-43d8-972a-34cb1038b5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:59.691630 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32678a83-4ef6-43d8-972a-34cb1038b5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:59.691918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32678a83-4ef6-43d8-972a-34cb1038b5c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:59.692569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32678a83-4ef6-43d8-972a-34cb1038b5c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:39:59.692904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32678a83-4ef6-43d8-972a-34cb1038b5c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:59.693064 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1979a2f-309f-4e7c-be69-44a8f9152fae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:59.694384 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1979a2f-309f-4e7c-be69-44a8f9152fae service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:59.694586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32678a83-4ef6-43d8-972a-34cb1038b5c9 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:59.703912 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1246/2497] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:39:59.708082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1252/2498] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:39:59.712149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1979a2f-309f-4e7c-be69-44a8f9152fae service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:59.722621 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1979a2f-309f-4e7c-be69-44a8f9152fae service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:59.728683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee377a0d-cd75-467a-964a-379af3414cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:59.729393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee377a0d-cd75-467a-964a-379af3414cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:59.729689 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee377a0d-cd75-467a-964a-379af3414cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:59.729941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee377a0d-cd75-467a-964a-379af3414cb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:59.730772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee377a0d-cd75-467a-964a-379af3414cb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:59.733175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee377a0d-cd75-467a-964a-379af3414cb9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:59.763656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1247/2499] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:39:59 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:39:59.768030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32678a83-4ef6-43d8-972a-34cb1038b5c9 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:59.779275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32678a83-4ef6-43d8-972a-34cb1038b5c9 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:59.839850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9b7ac-b0e7-4c59-8178-05e8805a4b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:59.840468 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9b7ac-b0e7-4c59-8178-05e8805a4b78 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:59.840783 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9b7ac-b0e7-4c59-8178-05e8805a4b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:59.841051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9b7ac-b0e7-4c59-8178-05e8805a4b78 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:59.841675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9b7ac-b0e7-4c59-8178-05e8805a4b78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:59.843629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9b7ac-b0e7-4c59-8178-05e8805a4b78 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:59.873163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1253/2500] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:39:59 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:39:59.874428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee377a0d-cd75-467a-964a-379af3414cb9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:59.885410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee377a0d-cd75-467a-964a-379af3414cb9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:59.890122 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc77a6-5934-48eb-831f-fef0d7c37e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:59.890725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc77a6-5934-48eb-831f-fef0d7c37e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:59.891008 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc77a6-5934-48eb-831f-fef0d7c37e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:59.891284 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc77a6-5934-48eb-831f-fef0d7c37e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:59.891924 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc77a6-5934-48eb-831f-fef0d7c37e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:59.893598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc77a6-5934-48eb-831f-fef0d7c37e29 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:59.905133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1248/2501] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:39:59.909285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9b7ac-b0e7-4c59-8178-05e8805a4b78 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:59.920198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9b7ac-b0e7-4c59-8178-05e8805a4b78 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:59.925418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa671b-df71-4551-83ae-882526a2355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:59.926035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa671b-df71-4551-83ae-882526a2355f admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:59.926313 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa671b-df71-4551-83ae-882526a2355f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:59.926583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa671b-df71-4551-83ae-882526a2355f admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:59.927244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa671b-df71-4551-83ae-882526a2355f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:59.928860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa671b-df71-4551-83ae-882526a2355f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:59.940777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1254/2502] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:39:59.946639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc77a6-5934-48eb-831f-fef0d7c37e29 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:59.956871 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc77a6-5934-48eb-831f-fef0d7c37e29 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:39:59.962504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db26bac-5125-4540-a86c-b02ef9a752b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:39:59.963163 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26bac-5125-4540-a86c-b02ef9a752b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:39:59.963454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26bac-5125-4540-a86c-b02ef9a752b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:39:59.963730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26bac-5125-4540-a86c-b02ef9a752b5 admin admin] PATH_INFO: `/v3/projects/ddc361addd644979aa650772ea10853e/users/46f0d7f467fd4457a31a44a68cc6af37/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:39:59.979780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db26bac-5125-4540-a86c-b02ef9a752b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddc361addd644979aa650772ea10853e, user_id=46f0d7f467fd4457a31a44a68cc6af37, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:39:59.982611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db26bac-5125-4540-a86c-b02ef9a752b5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:39:59.984094 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa671b-df71-4551-83ae-882526a2355f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:39:59.986736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e8fa671b-df71-4551-83ae-882526a2355f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:39:59.998289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1249/2503] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:39:59 2022] PUT /identity/v3/projects/ddc361addd644979aa650772ea10853e/users/46f0d7f467fd4457a31a44a68cc6af37/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:00.003773 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa671b-df71-4551-83ae-882526a2355f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:00.010280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bbe98-8025-454c-94b3-26c12bc1de31 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:00.010580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bbe98-8025-454c-94b3-26c12bc1de31 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:00.010859 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bbe98-8025-454c-94b3-26c12bc1de31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:00.032393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:00.042420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0a8bbe98-8025-454c-94b3-26c12bc1de31 None None] MFA Rules not processed for user `46f0d7f467fd4457a31a44a68cc6af37`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:00.043646 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0a8bbe98-8025-454c-94b3-26c12bc1de31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:00.064401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:00.065052 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:00.065329 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:00.065812 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] PATH_INFO: `/v3/projects/8bd3a3c6fa9044868cee42d4e57872d1` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:00.066721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bd3a3c6fa9044868cee42d4e57872d1)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:00.068890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:00.110526 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:00.115460 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1250/2504] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:40:00.119158 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bd3a3c6fa9044868cee42d4e57872d1', 'request_id': 'req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.126851 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bd3a3c6fa9044868cee42d4e57872d1', 'request_id': 'req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.127997 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bbe98-8025-454c-94b3-26c12bc1de31 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:00.129669 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0a8bbe98-8025-454c-94b3-26c12bc1de31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:00.133195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bd3a3c6fa9044868cee42d4e57872d1', 'request_id': 'req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.146304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] The token cache is being invalidate because project 8bd3a3c6fa9044868cee42d4e57872d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:00.146686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.147340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.149783 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1255/2505] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:39:59 2022] DELETE /identity/v3/projects/8bd3a3c6fa9044868cee42d4e57872d1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:00.153580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:00.157724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:00.208454 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bbe98-8025-454c-94b3-26c12bc1de31 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:00.224416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16216a1e-ba45-4fe3-ae89-ab41fe1e19f8 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:00.230665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:00.231292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:00.231567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:00.231839 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] PATH_INFO: `/v3/users/a3df3461859a42e2a2e132bb1ad64e77` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:00.232988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3df3461859a42e2a2e132bb1ad64e77)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:00.234894 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:00.277818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e2528-d198-422d-927f-f6d56a78f1fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:00.278442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e2528-d198-422d-927f-f6d56a78f1fa service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:00.278728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e2528-d198-422d-927f-f6d56a78f1fa service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:00.279005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e2528-d198-422d-927f-f6d56a78f1fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:00.280932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3b6e2528-d198-422d-927f-f6d56a78f1fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:00.282678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3df3461859a42e2a2e132bb1ad64e77'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.283430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3df3461859a42e2a2e132bb1ad64e77'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.284176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3df3461859a42e2a2e132bb1ad64e77'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.292334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3df3461859a42e2a2e132bb1ad64e77'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.314961 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3df3461859a42e2a2e132bb1ad64e77'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.316981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:00.331041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1256/2506] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:00 2022] DELETE /identity/v3/users/a3df3461859a42e2a2e132bb1ad64e77 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:00.341803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e2528-d198-422d-927f-f6d56a78f1fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:00.343415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e2528-d198-422d-927f-f6d56a78f1fa service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:00.353367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1251/2507] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:00.641922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:00.643751 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:00.719183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:00.745028 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94c062-ebfc-4bca-8f76-a470a5488eb5 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:00.753963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b531581-6529-4ba4-bae4-5619c7bc2778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:00.754512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b531581-6529-4ba4-bae4-5619c7bc2778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:00.754809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b531581-6529-4ba4-bae4-5619c7bc2778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:00.755079 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b531581-6529-4ba4-bae4-5619c7bc2778 admin admin] PATH_INFO: `/v3/projects/2844cf567c3340d1af281c7910cb590f` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:00.756206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b531581-6529-4ba4-bae4-5619c7bc2778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2844cf567c3340d1af281c7910cb590f)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:00.762154 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b531581-6529-4ba4-bae4-5619c7bc2778 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:00.815505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9b531581-6529-4ba4-bae4-5619c7bc2778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2844cf567c3340d1af281c7910cb590f', 'request_id': 'req-9b531581-6529-4ba4-bae4-5619c7bc2778'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.822470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9b531581-6529-4ba4-bae4-5619c7bc2778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2844cf567c3340d1af281c7910cb590f', 'request_id': 'req-9b531581-6529-4ba4-bae4-5619c7bc2778'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.827880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9b531581-6529-4ba4-bae4-5619c7bc2778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2844cf567c3340d1af281c7910cb590f', 'request_id': 'req-9b531581-6529-4ba4-bae4-5619c7bc2778'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.840627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9b531581-6529-4ba4-bae4-5619c7bc2778 admin admin] The token cache is being invalidate because project 2844cf567c3340d1af281c7910cb590f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:00.840943 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9b531581-6529-4ba4-bae4-5619c7bc2778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.841592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9b531581-6529-4ba4-bae4-5619c7bc2778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:00.843117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1257/2508] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:00 2022] DELETE /identity/v3/projects/2844cf567c3340d1af281c7910cb590f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:00.848992 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3747714-3c45-456d-affe-456a7d896f09 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:00.849307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3747714-3c45-456d-affe-456a7d896f09 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:00.849565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3747714-3c45-456d-affe-456a7d896f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:00.869928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d3747714-3c45-456d-affe-456a7d896f09 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:00.871141 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d3747714-3c45-456d-affe-456a7d896f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:00.929904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1252/2509] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:00.937928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c35fe-cd0b-4c44-82f3-894258698fb0 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:00.938239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c35fe-cd0b-4c44-82f3-894258698fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:00.938519 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c35fe-cd0b-4c44-82f3-894258698fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:00.960352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-819c35fe-cd0b-4c44-82f3-894258698fb0 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:00.961686 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-819c35fe-cd0b-4c44-82f3-894258698fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:00.969801 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1258/2510] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:00.974061 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3747714-3c45-456d-affe-456a7d896f09 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:00.984100 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3747714-3c45-456d-affe-456a7d896f09 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:00.988746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be71ca-d2f4-4902-85f7-0d37ec8ba053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:00.989378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be71ca-d2f4-4902-85f7-0d37ec8ba053 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:00.989662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be71ca-d2f4-4902-85f7-0d37ec8ba053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:00.989940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be71ca-d2f4-4902-85f7-0d37ec8ba053 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:00.990589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be71ca-d2f4-4902-85f7-0d37ec8ba053 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:40:00.990896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be71ca-d2f4-4902-85f7-0d37ec8ba053 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:00.993183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be71ca-d2f4-4902-85f7-0d37ec8ba053 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:01.004827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1253/2511] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:40:01.009903 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c35fe-cd0b-4c44-82f3-894258698fb0 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:01.019065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c35fe-cd0b-4c44-82f3-894258698fb0 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:01.024031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606504b-944b-4cb7-a09b-c31de8180404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:01.024628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606504b-944b-4cb7-a09b-c31de8180404 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:01.024943 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606504b-944b-4cb7-a09b-c31de8180404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:01.025222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606504b-944b-4cb7-a09b-c31de8180404 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:01.025858 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606504b-944b-4cb7-a09b-c31de8180404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:01.027802 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606504b-944b-4cb7-a09b-c31de8180404 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:01.056227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1259/2512] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:01 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:01.060939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be71ca-d2f4-4902-85f7-0d37ec8ba053 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:01.072070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be71ca-d2f4-4902-85f7-0d37ec8ba053 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:01.128574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ad0ec-cdb1-4fbb-88e2-161f2a597f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:01.129132 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ad0ec-cdb1-4fbb-88e2-161f2a597f58 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:01.129413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ad0ec-cdb1-4fbb-88e2-161f2a597f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:01.129687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ad0ec-cdb1-4fbb-88e2-161f2a597f58 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:01.130299 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ad0ec-cdb1-4fbb-88e2-161f2a597f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:01.132228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ad0ec-cdb1-4fbb-88e2-161f2a597f58 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:01.153554 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1254/2513] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:01 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:40:01.157762 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606504b-944b-4cb7-a09b-c31de8180404 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:01.169411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606504b-944b-4cb7-a09b-c31de8180404 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:01.174087 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c23415-65da-4c47-8678-228e4f295518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:01.174683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c23415-65da-4c47-8678-228e4f295518 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:01.174963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c23415-65da-4c47-8678-228e4f295518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:01.175235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c23415-65da-4c47-8678-228e4f295518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:01.175865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c23415-65da-4c47-8678-228e4f295518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:01.177506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c23415-65da-4c47-8678-228e4f295518 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:01.187734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1260/2514] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:01.192099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ad0ec-cdb1-4fbb-88e2-161f2a597f58 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:01.203684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ad0ec-cdb1-4fbb-88e2-161f2a597f58 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:01.208979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0408ad-04af-4234-a1f9-25f0a95d8fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:01.209628 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0408ad-04af-4234-a1f9-25f0a95d8fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:01.209919 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0408ad-04af-4234-a1f9-25f0a95d8fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:01.210202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0408ad-04af-4234-a1f9-25f0a95d8fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:01.211417 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0408ad-04af-4234-a1f9-25f0a95d8fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:01.213353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0408ad-04af-4234-a1f9-25f0a95d8fd1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:01.227003 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1255/2515] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:01.231411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c23415-65da-4c47-8678-228e4f295518 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:01.241349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c23415-65da-4c47-8678-228e4f295518 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:01.245966 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd16300-67be-4e22-abf7-bcfda53664b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:01.246616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd16300-67be-4e22-abf7-bcfda53664b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:01.246895 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd16300-67be-4e22-abf7-bcfda53664b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:01.247166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd16300-67be-4e22-abf7-bcfda53664b1 admin admin] PATH_INFO: `/v3/projects/e9b773b032444be2b94f3d6ec241f800/users/c4b395961baa4e6dbcefa46e45145a45/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:01.262040 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd16300-67be-4e22-abf7-bcfda53664b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b773b032444be2b94f3d6ec241f800, user_id=c4b395961baa4e6dbcefa46e45145a45, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:01.264694 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd16300-67be-4e22-abf7-bcfda53664b1 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:01.277128 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1261/2516] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:01 2022] PUT /identity/v3/projects/e9b773b032444be2b94f3d6ec241f800/users/c4b395961baa4e6dbcefa46e45145a45/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:01.287219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720be08-6b46-441e-b9ad-77e9fd6a1e40 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:01.287510 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720be08-6b46-441e-b9ad-77e9fd6a1e40 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:01.287787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720be08-6b46-441e-b9ad-77e9fd6a1e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:01.315120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c720be08-6b46-441e-b9ad-77e9fd6a1e40 None None] MFA Rules not processed for user `c4b395961baa4e6dbcefa46e45145a45`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:01.316335 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c720be08-6b46-441e-b9ad-77e9fd6a1e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:01.350667 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:01.385415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1256/2517] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:01.396246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd16300-67be-4e22-abf7-bcfda53664b1 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:01.398668 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-edd16300-67be-4e22-abf7-bcfda53664b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:01.456470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd16300-67be-4e22-abf7-bcfda53664b1 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:01.460146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03d12c-9589-4eaa-b2b0-f6fe274bdd01 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '748ad823b944441d907994cda6d8ac7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:01.461390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d12c-9589-4eaa-b2b0-f6fe274bdd01 service glance] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:01.461667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d12c-9589-4eaa-b2b0-f6fe274bdd01 service glance] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:01.462015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d12c-9589-4eaa-b2b0-f6fe274bdd01 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:01.470378 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03d12c-9589-4eaa-b2b0-f6fe274bdd01 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:01.471803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03d12c-9589-4eaa-b2b0-f6fe274bdd01 service glance] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:01.480634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1262/2518] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:01.856995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720be08-6b46-441e-b9ad-77e9fd6a1e40 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:01.858908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c720be08-6b46-441e-b9ad-77e9fd6a1e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:01.954359 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720be08-6b46-441e-b9ad-77e9fd6a1e40 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:01.959494 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912f15f-0b89-464e-a6a0-cfebab9193be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:01.959494 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912f15f-0b89-464e-a6a0-cfebab9193be service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:01.959494 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912f15f-0b89-464e-a6a0-cfebab9193be service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:01.959494 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912f15f-0b89-464e-a6a0-cfebab9193be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:01.961512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3912f15f-0b89-464e-a6a0-cfebab9193be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:02.031317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912f15f-0b89-464e-a6a0-cfebab9193be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:02.033239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912f15f-0b89-464e-a6a0-cfebab9193be service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:02.046501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1257/2519] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:05.371844 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03d12c-9589-4eaa-b2b0-f6fe274bdd01 service glance] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:05.374464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8a03d12c-9589-4eaa-b2b0-f6fe274bdd01 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:05.399790 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03d12c-9589-4eaa-b2b0-f6fe274bdd01 service glance] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:05.408139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2864691-fe3e-4af1-a801-99b30c68176d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:05.409086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2864691-fe3e-4af1-a801-99b30c68176d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:05.409533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2864691-fe3e-4af1-a801-99b30c68176d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:05.410149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2864691-fe3e-4af1-a801-99b30c68176d admin admin] PATH_INFO: `/v3/users/e09fd34737af41dc972d806dc3b0705e` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:05.411814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2864691-fe3e-4af1-a801-99b30c68176d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e09fd34737af41dc972d806dc3b0705e)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:05.414669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2864691-fe3e-4af1-a801-99b30c68176d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:05.432665 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:05.485571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d2864691-fe3e-4af1-a801-99b30c68176d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e09fd34737af41dc972d806dc3b0705e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:05.486297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d2864691-fe3e-4af1-a801-99b30c68176d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e09fd34737af41dc972d806dc3b0705e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:05.486957 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d2864691-fe3e-4af1-a801-99b30c68176d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e09fd34737af41dc972d806dc3b0705e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:05.498351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d2864691-fe3e-4af1-a801-99b30c68176d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e09fd34737af41dc972d806dc3b0705e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:05.514145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d2864691-fe3e-4af1-a801-99b30c68176d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e09fd34737af41dc972d806dc3b0705e'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:05.531588 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1263/2520] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:05 2022] DELETE /identity/v3/users/e09fd34737af41dc972d806dc3b0705e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:05.728918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912f15f-0b89-464e-a6a0-cfebab9193be service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:05.742763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912f15f-0b89-464e-a6a0-cfebab9193be service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:05.814289 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:05.814874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:05.815153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:05.815430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b admin admin] PATH_INFO: `/v3/projects/d22d9f4004394e1cbcd0a2afffd114b1` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:05.816647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22d9f4004394e1cbcd0a2afffd114b1)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:05.818623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:05.844973 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:05.868237 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22d9f4004394e1cbcd0a2afffd114b1', 'request_id': 'req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:05.875817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22d9f4004394e1cbcd0a2afffd114b1', 'request_id': 'req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:05.882407 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22d9f4004394e1cbcd0a2afffd114b1', 'request_id': 'req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:05.896261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b admin admin] The token cache is being invalidate because project d22d9f4004394e1cbcd0a2afffd114b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:05.896648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:05.897380 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b92eda05-3fc7-4bf6-a6b8-fefcdabeb88b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:05.899142 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1258/2521] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:05 2022] DELETE /identity/v3/projects/d22d9f4004394e1cbcd0a2afffd114b1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:05.904345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca715eb1-5e9a-4c11-bb6d-0889d28ad285 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:05.904631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca715eb1-5e9a-4c11-bb6d-0889d28ad285 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:05.904938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca715eb1-5e9a-4c11-bb6d-0889d28ad285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:05.924610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ca715eb1-5e9a-4c11-bb6d-0889d28ad285 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:05.925870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ca715eb1-5e9a-4c11-bb6d-0889d28ad285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:05.989254 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1264/2522] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:05.998201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61062-6b36-416f-93b1-0d6062a09022 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:05.998491 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61062-6b36-416f-93b1-0d6062a09022 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:05.998768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61062-6b36-416f-93b1-0d6062a09022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.019023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-31d61062-6b36-416f-93b1-0d6062a09022 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:06.020234 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-31d61062-6b36-416f-93b1-0d6062a09022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:06.028171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1259/2523] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:06.032652 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca715eb1-5e9a-4c11-bb6d-0889d28ad285 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:06.043609 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca715eb1-5e9a-4c11-bb6d-0889d28ad285 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:06.048259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ba434-e195-4ec1-aaef-260ee59d7d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:06.048951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ba434-e195-4ec1-aaef-260ee59d7d76 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.049657 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ba434-e195-4ec1-aaef-260ee59d7d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.051214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ba434-e195-4ec1-aaef-260ee59d7d76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.051871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ba434-e195-4ec1-aaef-260ee59d7d76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:40:06.052172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ba434-e195-4ec1-aaef-260ee59d7d76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:06.053800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ba434-e195-4ec1-aaef-260ee59d7d76 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:06.066391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1265/2524] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:40:06.071307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d61062-6b36-416f-93b1-0d6062a09022 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:06.081002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d61062-6b36-416f-93b1-0d6062a09022 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:06.085663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f55e6-1893-4bfc-bfc3-1b5ca42fb637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:06.086319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f55e6-1893-4bfc-bfc3-1b5ca42fb637 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.086597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f55e6-1893-4bfc-bfc3-1b5ca42fb637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.086867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f55e6-1893-4bfc-bfc3-1b5ca42fb637 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.087994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f55e6-1893-4bfc-bfc3-1b5ca42fb637 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:06.089909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f55e6-1893-4bfc-bfc3-1b5ca42fb637 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:06.117401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1260/2525] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:06 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:40:06.121767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ba434-e195-4ec1-aaef-260ee59d7d76 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:06.133246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ba434-e195-4ec1-aaef-260ee59d7d76 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:06.201756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820bb6-28e3-4bc9-8572-0a3833e63f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:06.202476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820bb6-28e3-4bc9-8572-0a3833e63f43 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.202810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820bb6-28e3-4bc9-8572-0a3833e63f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.203155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820bb6-28e3-4bc9-8572-0a3833e63f43 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.203873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22820bb6-28e3-4bc9-8572-0a3833e63f43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:06.205960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22820bb6-28e3-4bc9-8572-0a3833e63f43 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:06.230680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1266/2526] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:06 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:06.234837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f55e6-1893-4bfc-bfc3-1b5ca42fb637 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:06.245834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f55e6-1893-4bfc-bfc3-1b5ca42fb637 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:06.250666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81372f8-a7f1-44c9-aefd-717346d231f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:06.251290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81372f8-a7f1-44c9-aefd-717346d231f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.251576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81372f8-a7f1-44c9-aefd-717346d231f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.251852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81372f8-a7f1-44c9-aefd-717346d231f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.252516 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81372f8-a7f1-44c9-aefd-717346d231f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:06.254193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81372f8-a7f1-44c9-aefd-717346d231f7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:06.267703 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1261/2527] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:06.269401 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820bb6-28e3-4bc9-8572-0a3833e63f43 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:06.278043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:06.282109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820bb6-28e3-4bc9-8572-0a3833e63f43 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:06.284581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81372f8-a7f1-44c9-aefd-717346d231f7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:06.286546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a81372f8-a7f1-44c9-aefd-717346d231f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:06.288341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03f81c-946b-4996-9403-426700cce050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:06.288884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03f81c-946b-4996-9403-426700cce050 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.289650 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03f81c-946b-4996-9403-426700cce050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.290310 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03f81c-946b-4996-9403-426700cce050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.292089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03f81c-946b-4996-9403-426700cce050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:06.293761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03f81c-946b-4996-9403-426700cce050 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:06.306589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1267/2528] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:06.310755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03f81c-946b-4996-9403-426700cce050 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:06.321351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03f81c-946b-4996-9403-426700cce050 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:06.326875 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b800b-9bf0-43ac-ab69-5aebbd3b4f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:06.327540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b800b-9bf0-43ac-ab69-5aebbd3b4f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.327816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b800b-9bf0-43ac-ab69-5aebbd3b4f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.328116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b800b-9bf0-43ac-ab69-5aebbd3b4f79 admin admin] PATH_INFO: `/v3/projects/846c48df6b6045a1b1215bcfb3ebb637/users/ac2ea6ec3fe640e182fa689af0ec6f3c/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.344111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b800b-9bf0-43ac-ab69-5aebbd3b4f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=846c48df6b6045a1b1215bcfb3ebb637, user_id=ac2ea6ec3fe640e182fa689af0ec6f3c, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:06.346926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b800b-9bf0-43ac-ab69-5aebbd3b4f79 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:06.361918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1268/2529] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:06 2022] PUT /identity/v3/projects/846c48df6b6045a1b1215bcfb3ebb637/users/ac2ea6ec3fe640e182fa689af0ec6f3c/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:06.366511 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81372f8-a7f1-44c9-aefd-717346d231f7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:06.372919 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779e0dd-c6f2-4f28-8ae8-8ed36d2905b8 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.373413 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779e0dd-c6f2-4f28-8ae8-8ed36d2905b8 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.373413 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779e0dd-c6f2-4f28-8ae8-8ed36d2905b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.403721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-3779e0dd-c6f2-4f28-8ae8-8ed36d2905b8 None None] MFA Rules not processed for user `ac2ea6ec3fe640e182fa689af0ec6f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:06.406812 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3779e0dd-c6f2-4f28-8ae8-8ed36d2905b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:06.430266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ffb8c-9789-42aa-9890-42dbac960398 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:06.430908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ffb8c-9789-42aa-9890-42dbac960398 service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.431281 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ffb8c-9789-42aa-9890-42dbac960398 service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.431579 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ffb8c-9789-42aa-9890-42dbac960398 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.434890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c60ffb8c-9789-42aa-9890-42dbac960398 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:06.449367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ffb8c-9789-42aa-9890-42dbac960398 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:06.451407 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ffb8c-9789-42aa-9890-42dbac960398 service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:06.466765 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1262/2530] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:40:06.488779 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1269/2531] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:40:06.492143 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ffb8c-9789-42aa-9890-42dbac960398 service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:06.498307 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:06.577269 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ffb8c-9789-42aa-9890-42dbac960398 service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:06.582475 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdec44-1243-4c1e-86e6-e7d6ecee1064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:06.583082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdec44-1243-4c1e-86e6-e7d6ecee1064 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.583365 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdec44-1243-4c1e-86e6-e7d6ecee1064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.583629 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdec44-1243-4c1e-86e6-e7d6ecee1064 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.584258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdec44-1243-4c1e-86e6-e7d6ecee1064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:06.586195 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdec44-1243-4c1e-86e6-e7d6ecee1064 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:06.618214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1263/2532] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:06 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:40:06.622875 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779e0dd-c6f2-4f28-8ae8-8ed36d2905b8 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:06.633846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779e0dd-c6f2-4f28-8ae8-8ed36d2905b8 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:06.701693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a031e-0831-4fac-8dc0-96455e15397f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:06.702402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a031e-0831-4fac-8dc0-96455e15397f admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.702685 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a031e-0831-4fac-8dc0-96455e15397f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.703037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a031e-0831-4fac-8dc0-96455e15397f admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.703716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a031e-0831-4fac-8dc0-96455e15397f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:06.705818 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a031e-0831-4fac-8dc0-96455e15397f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:06.728956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1270/2533] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:06 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:06.733386 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdec44-1243-4c1e-86e6-e7d6ecee1064 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:06.744338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdec44-1243-4c1e-86e6-e7d6ecee1064 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:06.749627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f60488-dcfb-422d-ad60-d7b819c89232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:06.750396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f60488-dcfb-422d-ad60-d7b819c89232 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.750688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f60488-dcfb-422d-ad60-d7b819c89232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.750994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f60488-dcfb-422d-ad60-d7b819c89232 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.751748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f60488-dcfb-422d-ad60-d7b819c89232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:06.753581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f60488-dcfb-422d-ad60-d7b819c89232 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:06.768410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1264/2534] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:06.772523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a031e-0831-4fac-8dc0-96455e15397f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:06.783061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a031e-0831-4fac-8dc0-96455e15397f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:06.787644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4d293-28b0-482e-978a-2f1122b83422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:06.788230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4d293-28b0-482e-978a-2f1122b83422 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.788506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4d293-28b0-482e-978a-2f1122b83422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.788794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4d293-28b0-482e-978a-2f1122b83422 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.789423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4d293-28b0-482e-978a-2f1122b83422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:06.791248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4d293-28b0-482e-978a-2f1122b83422 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:06.801449 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1271/2535] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:06.805948 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f60488-dcfb-422d-ad60-d7b819c89232 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:06.817032 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f60488-dcfb-422d-ad60-d7b819c89232 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:06.821624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68723b-2a07-4717-9716-16282db802b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:06.822926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68723b-2a07-4717-9716-16282db802b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.823247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68723b-2a07-4717-9716-16282db802b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.823513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68723b-2a07-4717-9716-16282db802b6 admin admin] PATH_INFO: `/v3/projects/c5c73e49194b40c49f89eac4560a4286/users/ef166afde72a4222b858f474d66ecf9a/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.838547 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68723b-2a07-4717-9716-16282db802b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c73e49194b40c49f89eac4560a4286, user_id=ef166afde72a4222b858f474d66ecf9a, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:06.841224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68723b-2a07-4717-9716-16282db802b6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:06.854405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1265/2536] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:06 2022] PUT /identity/v3/projects/c5c73e49194b40c49f89eac4560a4286/users/ef166afde72a4222b858f474d66ecf9a/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:06.866074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d2ddb-d80e-40ab-90c9-5e71eba233ed None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:06.866392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d2ddb-d80e-40ab-90c9-5e71eba233ed None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:06.866668 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d2ddb-d80e-40ab-90c9-5e71eba233ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:06.906442 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-cc5d2ddb-d80e-40ab-90c9-5e71eba233ed None None] MFA Rules not processed for user `ef166afde72a4222b858f474d66ecf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:06.907683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-cc5d2ddb-d80e-40ab-90c9-5e71eba233ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:06.934481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:06.986022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1272/2537] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:06.990677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68723b-2a07-4717-9716-16282db802b6 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:07.054794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68723b-2a07-4717-9716-16282db802b6 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:07.060085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788f9f9-5df0-4392-9c83-1a641c1df5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:07.060688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788f9f9-5df0-4392-9c83-1a641c1df5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:07.061010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788f9f9-5df0-4392-9c83-1a641c1df5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:07.061283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788f9f9-5df0-4392-9c83-1a641c1df5e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:07.061918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788f9f9-5df0-4392-9c83-1a641c1df5e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:07.064102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788f9f9-5df0-4392-9c83-1a641c1df5e0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:07.095660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1266/2538] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:06 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:07.096975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d2ddb-d80e-40ab-90c9-5e71eba233ed None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:07.108974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d2ddb-d80e-40ab-90c9-5e71eba233ed None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:07.168580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5c1ed-84fa-4fcc-9182-97482edc49cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:07.169139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5c1ed-84fa-4fcc-9182-97482edc49cc admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:07.169421 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5c1ed-84fa-4fcc-9182-97482edc49cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:07.169695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5c1ed-84fa-4fcc-9182-97482edc49cc admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:07.170320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5c1ed-84fa-4fcc-9182-97482edc49cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:07.172227 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5c1ed-84fa-4fcc-9182-97482edc49cc admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:07.195945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1273/2539] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:07 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:07.201451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788f9f9-5df0-4392-9c83-1a641c1df5e0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:07.214301 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788f9f9-5df0-4392-9c83-1a641c1df5e0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:07.220993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e1810-9bb4-46ed-b596-348bd305d902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:07.221509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e1810-9bb4-46ed-b596-348bd305d902 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:07.221791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e1810-9bb4-46ed-b596-348bd305d902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:07.222068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e1810-9bb4-46ed-b596-348bd305d902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:07.222722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e1810-9bb4-46ed-b596-348bd305d902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:07.224384 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e1810-9bb4-46ed-b596-348bd305d902 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:07.236935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1267/2540] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:07.241238 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5c1ed-84fa-4fcc-9182-97482edc49cc admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:07.252433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5c1ed-84fa-4fcc-9182-97482edc49cc admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:07.257166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007eeac4-a78d-4b9c-9474-403eeefe38c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:07.257823 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007eeac4-a78d-4b9c-9474-403eeefe38c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:07.258101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007eeac4-a78d-4b9c-9474-403eeefe38c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:07.258372 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007eeac4-a78d-4b9c-9474-403eeefe38c0 admin admin] PATH_INFO: `/v3/projects/2c431786e54f4a4d8427fe183db8c443/users/8fdddd41b1db4a21a32f77f678bbebdb/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:07.273937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007eeac4-a78d-4b9c-9474-403eeefe38c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c431786e54f4a4d8427fe183db8c443, user_id=8fdddd41b1db4a21a32f77f678bbebdb, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:07.276688 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007eeac4-a78d-4b9c-9474-403eeefe38c0 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:07.291194 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1274/2541] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:07 2022] PUT /identity/v3/projects/2c431786e54f4a4d8427fe183db8c443/users/8fdddd41b1db4a21a32f77f678bbebdb/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:07.295487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e1810-9bb4-46ed-b596-348bd305d902 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:07.305975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e1810-9bb4-46ed-b596-348bd305d902 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:07.334097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:07.366650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442a9c1-7aa2-4a9d-8c8b-535251b766a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:07.367266 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442a9c1-7aa2-4a9d-8c8b-535251b766a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:07.367544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442a9c1-7aa2-4a9d-8c8b-535251b766a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:07.367817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442a9c1-7aa2-4a9d-8c8b-535251b766a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:07.368563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442a9c1-7aa2-4a9d-8c8b-535251b766a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:07.370819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442a9c1-7aa2-4a9d-8c8b-535251b766a5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:07.381227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1268/2542] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:07.385815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007eeac4-a78d-4b9c-9474-403eeefe38c0 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:07.396262 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007eeac4-a78d-4b9c-9474-403eeefe38c0 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:07.400860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575b948-fa06-43d7-a386-4be05d5b3d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:07.401514 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575b948-fa06-43d7-a386-4be05d5b3d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:07.401813 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575b948-fa06-43d7-a386-4be05d5b3d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:07.402085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575b948-fa06-43d7-a386-4be05d5b3d96 admin admin] PATH_INFO: `/v3/projects/2c431786e54f4a4d8427fe183db8c443/users/8fdddd41b1db4a21a32f77f678bbebdb/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:07.404045 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575b948-fa06-43d7-a386-4be05d5b3d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c431786e54f4a4d8427fe183db8c443, user_id=8fdddd41b1db4a21a32f77f678bbebdb, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:07.406736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575b948-fa06-43d7-a386-4be05d5b3d96 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:07.417954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1275/2543] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:07 2022] PUT /identity/v3/projects/2c431786e54f4a4d8427fe183db8c443/users/8fdddd41b1db4a21a32f77f678bbebdb/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:07.428306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6ce74-03a9-4895-8e5d-49a905396ec9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:07.428531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6ce74-03a9-4895-8e5d-49a905396ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:07.428894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6ce74-03a9-4895-8e5d-49a905396ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:07.457463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d7a6ce74-03a9-4895-8e5d-49a905396ec9 None None] MFA Rules not processed for user `8fdddd41b1db4a21a32f77f678bbebdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:07.458685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d7a6ce74-03a9-4895-8e5d-49a905396ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:07.528955 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1269/2544] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:07.539716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575b948-fa06-43d7-a386-4be05d5b3d96 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:07.542026 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b575b948-fa06-43d7-a386-4be05d5b3d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:07.600720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575b948-fa06-43d7-a386-4be05d5b3d96 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:07.605483 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3734c7-dea6-4557-8a4d-160e0d897aaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:07.606101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3734c7-dea6-4557-8a4d-160e0d897aaf service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:07.606383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3734c7-dea6-4557-8a4d-160e0d897aaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:07.606656 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3734c7-dea6-4557-8a4d-160e0d897aaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:07.616576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3734c7-dea6-4557-8a4d-160e0d897aaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:07.618035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3734c7-dea6-4557-8a4d-160e0d897aaf service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:07.631544 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1276/2545] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:08.163938 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6ce74-03a9-4895-8e5d-49a905396ec9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:08.173828 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6ce74-03a9-4895-8e5d-49a905396ec9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:08.177431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107c084-6a6f-4bb9-86e5-4fea49116d98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:08.178036 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107c084-6a6f-4bb9-86e5-4fea49116d98 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:08.178455 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107c084-6a6f-4bb9-86e5-4fea49116d98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:08.179257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107c084-6a6f-4bb9-86e5-4fea49116d98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:08.240392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107c084-6a6f-4bb9-86e5-4fea49116d98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:08.242326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107c084-6a6f-4bb9-86e5-4fea49116d98 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:08.252396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1270/2546] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:08.915043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3734c7-dea6-4557-8a4d-160e0d897aaf service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:08.915043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8f3734c7-dea6-4557-8a4d-160e0d897aaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:08.942581 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:08.984697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3734c7-dea6-4557-8a4d-160e0d897aaf service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:08.990348 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:08.991048 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:08.991332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:08.991624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] PATH_INFO: `/v3/users/46f0d7f467fd4457a31a44a68cc6af37` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:08.992987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f0d7f467fd4457a31a44a68cc6af37)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:08.995008 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:09.044986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f0d7f467fd4457a31a44a68cc6af37'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:09.045586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f0d7f467fd4457a31a44a68cc6af37'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:09.046211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f0d7f467fd4457a31a44a68cc6af37'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:09.053429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f0d7f467fd4457a31a44a68cc6af37'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:09.066115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f0d7f467fd4457a31a44a68cc6af37'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:09.080880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1277/2547] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:08 2022] DELETE /identity/v3/users/46f0d7f467fd4457a31a44a68cc6af37 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:09.085342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99befa0d-2646-41f9-a3f0-939840717c7f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:09.085660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99befa0d-2646-41f9-a3f0-939840717c7f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:09.085937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99befa0d-2646-41f9-a3f0-939840717c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:09.105049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-99befa0d-2646-41f9-a3f0-939840717c7f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:09.106280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-99befa0d-2646-41f9-a3f0-939840717c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:09.151687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:09.168160 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1271/2548] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:09.177890 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:09.179318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:09.261496 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306a7ad-3066-4fda-9d9c-d415e7301a3f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:09.265417 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def19b3-d347-4d9b-bb36-0efc3a7ee23f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:09.266042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def19b3-d347-4d9b-bb36-0efc3a7ee23f service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:09.266318 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def19b3-d347-4d9b-bb36-0efc3a7ee23f service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:09.266588 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def19b3-d347-4d9b-bb36-0efc3a7ee23f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:09.276164 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def19b3-d347-4d9b-bb36-0efc3a7ee23f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:09.277615 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def19b3-d347-4d9b-bb36-0efc3a7ee23f service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:09.286971 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1278/2549] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:40:09.566179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99befa0d-2646-41f9-a3f0-939840717c7f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:09.567728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-99befa0d-2646-41f9-a3f0-939840717c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:09.594614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99befa0d-2646-41f9-a3f0-939840717c7f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:09.608064 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467f660-e86a-4423-ba46-4d905cbc0544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:09.608872 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467f660-e86a-4423-ba46-4d905cbc0544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:09.609355 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467f660-e86a-4423-ba46-4d905cbc0544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:09.609726 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467f660-e86a-4423-ba46-4d905cbc0544 admin admin] PATH_INFO: `/v3/projects/ddc361addd644979aa650772ea10853e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:09.611149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467f660-e86a-4423-ba46-4d905cbc0544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddc361addd644979aa650772ea10853e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:09.613338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467f660-e86a-4423-ba46-4d905cbc0544 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:09.675067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c467f660-e86a-4423-ba46-4d905cbc0544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddc361addd644979aa650772ea10853e', 'request_id': 'req-c467f660-e86a-4423-ba46-4d905cbc0544'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:09.686039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c467f660-e86a-4423-ba46-4d905cbc0544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddc361addd644979aa650772ea10853e', 'request_id': 'req-c467f660-e86a-4423-ba46-4d905cbc0544'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:09.693080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c467f660-e86a-4423-ba46-4d905cbc0544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddc361addd644979aa650772ea10853e', 'request_id': 'req-c467f660-e86a-4423-ba46-4d905cbc0544'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:09.705993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c467f660-e86a-4423-ba46-4d905cbc0544 admin admin] The token cache is being invalidate because project ddc361addd644979aa650772ea10853e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:09.706317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c467f660-e86a-4423-ba46-4d905cbc0544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:09.706927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c467f660-e86a-4423-ba46-4d905cbc0544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:09.711038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1272/2550] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:09 2022] DELETE /identity/v3/projects/ddc361addd644979aa650772ea10853e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:09.714165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd89c0-ef8a-4ee7-802f-f297f99162e5 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:09.714539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd89c0-ef8a-4ee7-802f-f297f99162e5 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:09.715151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd89c0-ef8a-4ee7-802f-f297f99162e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:09.745706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-a3bd89c0-ef8a-4ee7-802f-f297f99162e5 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:09.747043 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a3bd89c0-ef8a-4ee7-802f-f297f99162e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:09.767397 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:09.814758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1279/2551] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:09.822811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ff10f-d626-4398-88d3-dff097c2fec1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:09.823167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ff10f-d626-4398-88d3-dff097c2fec1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:09.823514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ff10f-d626-4398-88d3-dff097c2fec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:09.844067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-870ff10f-d626-4398-88d3-dff097c2fec1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:09.845329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-870ff10f-d626-4398-88d3-dff097c2fec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:09.854131 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1273/2552] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:09.858507 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd89c0-ef8a-4ee7-802f-f297f99162e5 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:09.868441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd89c0-ef8a-4ee7-802f-f297f99162e5 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:09.873608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39c5a4-65ca-4206-8701-829f6e365675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:09.874198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39c5a4-65ca-4206-8701-829f6e365675 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:09.874478 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39c5a4-65ca-4206-8701-829f6e365675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:09.874777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39c5a4-65ca-4206-8701-829f6e365675 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:09.875425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39c5a4-65ca-4206-8701-829f6e365675 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:40:09.875728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39c5a4-65ca-4206-8701-829f6e365675 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:09.877428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39c5a4-65ca-4206-8701-829f6e365675 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:09.889260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1280/2553] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:40:09.893635 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ff10f-d626-4398-88d3-dff097c2fec1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:09.903522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ff10f-d626-4398-88d3-dff097c2fec1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:09.908555 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f22bb8-490f-402e-92dc-856706b17edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:09.909208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f22bb8-490f-402e-92dc-856706b17edb admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:09.909488 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f22bb8-490f-402e-92dc-856706b17edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:09.909763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f22bb8-490f-402e-92dc-856706b17edb admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:09.910403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f22bb8-490f-402e-92dc-856706b17edb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:09.912326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f22bb8-490f-402e-92dc-856706b17edb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:09.941171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1274/2554] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:09 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:09.945566 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39c5a4-65ca-4206-8701-829f6e365675 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:09.955586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39c5a4-65ca-4206-8701-829f6e365675 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:10.018053 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c2c1c-222a-4e49-bd79-e5b81cf82022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:10.018662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c2c1c-222a-4e49-bd79-e5b81cf82022 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:10.018945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c2c1c-222a-4e49-bd79-e5b81cf82022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:10.019222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c2c1c-222a-4e49-bd79-e5b81cf82022 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:10.019883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c2c1c-222a-4e49-bd79-e5b81cf82022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:10.021867 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c2c1c-222a-4e49-bd79-e5b81cf82022 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:10.045909 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1281/2555] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:09 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:10.049926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f22bb8-490f-402e-92dc-856706b17edb admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:10.060577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f22bb8-490f-402e-92dc-856706b17edb admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:10.065685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58fc7f-1f26-40f5-a27f-efc048fe41b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:10.066230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58fc7f-1f26-40f5-a27f-efc048fe41b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:10.066512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58fc7f-1f26-40f5-a27f-efc048fe41b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:10.066786 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58fc7f-1f26-40f5-a27f-efc048fe41b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:10.067425 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58fc7f-1f26-40f5-a27f-efc048fe41b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:10.069066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58fc7f-1f26-40f5-a27f-efc048fe41b8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:10.080139 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1275/2556] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:10.084667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c2c1c-222a-4e49-bd79-e5b81cf82022 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:10.095443 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c2c1c-222a-4e49-bd79-e5b81cf82022 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:10.100078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e596946-ca81-4ccb-a86f-f3209be81f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:10.100970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596946-ca81-4ccb-a86f-f3209be81f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:10.101270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596946-ca81-4ccb-a86f-f3209be81f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:10.101541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596946-ca81-4ccb-a86f-f3209be81f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:10.102175 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e596946-ca81-4ccb-a86f-f3209be81f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:10.104997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e596946-ca81-4ccb-a86f-f3209be81f5c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:10.243874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1282/2557] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:10.248074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58fc7f-1f26-40f5-a27f-efc048fe41b8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:10.258449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58fc7f-1f26-40f5-a27f-efc048fe41b8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:10.263778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da44e53-7669-4a38-8ad7-8f3a633f30e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:10.264461 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da44e53-7669-4a38-8ad7-8f3a633f30e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:10.264740 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da44e53-7669-4a38-8ad7-8f3a633f30e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:10.265038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da44e53-7669-4a38-8ad7-8f3a633f30e7 admin admin] PATH_INFO: `/v3/projects/dff5f7011c6b46fa89fcfa54a74f91d9/users/266b16b12a034e94876519eb492623e6/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:10.281838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da44e53-7669-4a38-8ad7-8f3a633f30e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff5f7011c6b46fa89fcfa54a74f91d9, user_id=266b16b12a034e94876519eb492623e6, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:10.284501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da44e53-7669-4a38-8ad7-8f3a633f30e7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:10.297241 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1276/2558] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:10 2022] PUT /identity/v3/projects/dff5f7011c6b46fa89fcfa54a74f91d9/users/266b16b12a034e94876519eb492623e6/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:10.308803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffb225-1a22-4baf-acfb-c67c772b12cd None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:10.309091 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffb225-1a22-4baf-acfb-c67c772b12cd None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:10.309364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffb225-1a22-4baf-acfb-c67c772b12cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:10.338935 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-66ffb225-1a22-4baf-acfb-c67c772b12cd None None] MFA Rules not processed for user `266b16b12a034e94876519eb492623e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:10.340061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-66ffb225-1a22-4baf-acfb-c67c772b12cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:10.410967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1283/2559] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:10.418610 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da44e53-7669-4a38-8ad7-8f3a633f30e7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:10.420354 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5da44e53-7669-4a38-8ad7-8f3a633f30e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:10.495478 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da44e53-7669-4a38-8ad7-8f3a633f30e7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:10.501116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e5939-40b9-4e41-96f1-b345c51f0588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:10.501728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e5939-40b9-4e41-96f1-b345c51f0588 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:10.502012 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e5939-40b9-4e41-96f1-b345c51f0588 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:10.502295 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e5939-40b9-4e41-96f1-b345c51f0588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:10.507401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:10.512799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e5939-40b9-4e41-96f1-b345c51f0588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:10.514766 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e5939-40b9-4e41-96f1-b345c51f0588 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:10.525622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1277/2560] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:10.737706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffb225-1a22-4baf-acfb-c67c772b12cd None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:10.759697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:10.807962 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffb225-1a22-4baf-acfb-c67c772b12cd None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:10.813321 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:10.814076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:10.814360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:10.814856 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] PATH_INFO: `/v3/users/266b16b12a034e94876519eb492623e6` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:10.816285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=266b16b12a034e94876519eb492623e6)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:10.823337 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:10.882009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '266b16b12a034e94876519eb492623e6'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:10.882658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '266b16b12a034e94876519eb492623e6'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:10.885471 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '266b16b12a034e94876519eb492623e6'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:10.895689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '266b16b12a034e94876519eb492623e6'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:10.911393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '266b16b12a034e94876519eb492623e6'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:10.927808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1284/2561] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:10 2022] DELETE /identity/v3/users/266b16b12a034e94876519eb492623e6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:10.932498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7ec12-9b47-4291-b777-7db649dbe047 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:10.933066 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7ec12-9b47-4291-b777-7db649dbe047 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:10.933152 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7ec12-9b47-4291-b777-7db649dbe047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:10.955179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-abd7ec12-9b47-4291-b777-7db649dbe047 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:10.956499 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-abd7ec12-9b47-4291-b777-7db649dbe047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:11.039046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1278/2562] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:11.055605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:11.055605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:11.132835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3dfd3-7034-414b-acb1-fb84ac67fee8 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:11.137296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaba3ed-c21b-4363-8ea8-df7b7388cafa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:11.137921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaba3ed-c21b-4363-8ea8-df7b7388cafa service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:11.138204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaba3ed-c21b-4363-8ea8-df7b7388cafa service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:11.139023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaba3ed-c21b-4363-8ea8-df7b7388cafa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:11.151755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaba3ed-c21b-4363-8ea8-df7b7388cafa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:11.153293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaba3ed-c21b-4363-8ea8-df7b7388cafa service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:11.165606 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1285/2563] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:11.480174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7ec12-9b47-4291-b777-7db649dbe047 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:11.481660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-abd7ec12-9b47-4291-b777-7db649dbe047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:11.495830 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7ec12-9b47-4291-b777-7db649dbe047 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:11.500942 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050a044-7636-45bf-a55f-3422deac3a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:11.501595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050a044-7636-45bf-a55f-3422deac3a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:11.501877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050a044-7636-45bf-a55f-3422deac3a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:11.502334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050a044-7636-45bf-a55f-3422deac3a76 admin admin] PATH_INFO: `/v3/projects/dff5f7011c6b46fa89fcfa54a74f91d9` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:11.503645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050a044-7636-45bf-a55f-3422deac3a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff5f7011c6b46fa89fcfa54a74f91d9)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:11.505829 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050a044-7636-45bf-a55f-3422deac3a76 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:11.560086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e050a044-7636-45bf-a55f-3422deac3a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff5f7011c6b46fa89fcfa54a74f91d9', 'request_id': 'req-e050a044-7636-45bf-a55f-3422deac3a76'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:11.566083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e050a044-7636-45bf-a55f-3422deac3a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff5f7011c6b46fa89fcfa54a74f91d9', 'request_id': 'req-e050a044-7636-45bf-a55f-3422deac3a76'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:11.574565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e050a044-7636-45bf-a55f-3422deac3a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff5f7011c6b46fa89fcfa54a74f91d9', 'request_id': 'req-e050a044-7636-45bf-a55f-3422deac3a76'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:11.589118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e050a044-7636-45bf-a55f-3422deac3a76 admin admin] The token cache is being invalidate because project dff5f7011c6b46fa89fcfa54a74f91d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:11.589415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e050a044-7636-45bf-a55f-3422deac3a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:11.590060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e050a044-7636-45bf-a55f-3422deac3a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:11.591566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1279/2564] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:11 2022] DELETE /identity/v3/projects/dff5f7011c6b46fa89fcfa54a74f91d9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:11.596374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16112d34-a322-45fb-ab2c-9ba41f97f612 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:11.596658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16112d34-a322-45fb-ab2c-9ba41f97f612 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:11.596973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16112d34-a322-45fb-ab2c-9ba41f97f612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:11.616854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-16112d34-a322-45fb-ab2c-9ba41f97f612 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:11.618070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-16112d34-a322-45fb-ab2c-9ba41f97f612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:11.659551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:11.682509 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1286/2565] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:11.691065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36de9d-c02e-454a-b6f4-03ae296478d1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:11.691363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36de9d-c02e-454a-b6f4-03ae296478d1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:11.691715 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36de9d-c02e-454a-b6f4-03ae296478d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:11.712317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1a36de9d-c02e-454a-b6f4-03ae296478d1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:11.713842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1a36de9d-c02e-454a-b6f4-03ae296478d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:11.722041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1280/2566] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:40:11.726322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16112d34-a322-45fb-ab2c-9ba41f97f612 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:11.737249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16112d34-a322-45fb-ab2c-9ba41f97f612 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:11.742573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe678890-3173-4417-8064-512d55fa2ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:11.743955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe678890-3173-4417-8064-512d55fa2ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:11.744424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe678890-3173-4417-8064-512d55fa2ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:11.744843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe678890-3173-4417-8064-512d55fa2ceb admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:11.746020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe678890-3173-4417-8064-512d55fa2ceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:40:11.746326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe678890-3173-4417-8064-512d55fa2ceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:11.747955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe678890-3173-4417-8064-512d55fa2ceb admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:11.760315 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1287/2567] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:40:11.764622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36de9d-c02e-454a-b6f4-03ae296478d1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:11.774311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36de9d-c02e-454a-b6f4-03ae296478d1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:11.779273 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d34379-a0df-40d4-ab44-49871f6444f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:11.779887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d34379-a0df-40d4-ab44-49871f6444f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:11.780164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d34379-a0df-40d4-ab44-49871f6444f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:11.780435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d34379-a0df-40d4-ab44-49871f6444f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:11.781087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d34379-a0df-40d4-ab44-49871f6444f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:11.782994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d34379-a0df-40d4-ab44-49871f6444f3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:11.813343 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1281/2568] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:11 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:11.817334 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe678890-3173-4417-8064-512d55fa2ceb admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:11.828484 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe678890-3173-4417-8064-512d55fa2ceb admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:11.888545 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc09ee2-62bb-4533-85e5-ef7e3b972211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:11.889652 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc09ee2-62bb-4533-85e5-ef7e3b972211 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:11.890172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc09ee2-62bb-4533-85e5-ef7e3b972211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:11.890676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc09ee2-62bb-4533-85e5-ef7e3b972211 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:11.891831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc09ee2-62bb-4533-85e5-ef7e3b972211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:11.895569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc09ee2-62bb-4533-85e5-ef7e3b972211 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:11.920267 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1288/2569] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:11 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:11.924359 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d34379-a0df-40d4-ab44-49871f6444f3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:11.936723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d34379-a0df-40d4-ab44-49871f6444f3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:11.941632 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb5695-cf80-4b25-bf31-674c375ab46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:11.942341 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb5695-cf80-4b25-bf31-674c375ab46a admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:11.942623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb5695-cf80-4b25-bf31-674c375ab46a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:11.942923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb5695-cf80-4b25-bf31-674c375ab46a admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:11.943557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb5695-cf80-4b25-bf31-674c375ab46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:11.945205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb5695-cf80-4b25-bf31-674c375ab46a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:11.955827 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1282/2570] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:11.960396 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc09ee2-62bb-4533-85e5-ef7e3b972211 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:11.970754 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc09ee2-62bb-4533-85e5-ef7e3b972211 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:11.975752 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c447b-677a-4090-946b-547de6b058d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:11.976349 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c447b-677a-4090-946b-547de6b058d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:11.976626 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c447b-677a-4090-946b-547de6b058d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:11.976936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c447b-677a-4090-946b-547de6b058d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:11.978063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c447b-677a-4090-946b-547de6b058d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:11.979656 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c447b-677a-4090-946b-547de6b058d3 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:11.990633 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1289/2571] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:11.995490 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb5695-cf80-4b25-bf31-674c375ab46a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:12.005601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb5695-cf80-4b25-bf31-674c375ab46a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:12.010391 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dff78a-e11e-4671-b3f3-8eeeab43dbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:12.011124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dff78a-e11e-4671-b3f3-8eeeab43dbb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:12.011446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dff78a-e11e-4671-b3f3-8eeeab43dbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:12.011775 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dff78a-e11e-4671-b3f3-8eeeab43dbb4 admin admin] PATH_INFO: `/v3/projects/c18a884cdb3c4b28976d5634ac693430/users/a76b340a2b8e46218719b06a20c733e9/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:12.027473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dff78a-e11e-4671-b3f3-8eeeab43dbb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c18a884cdb3c4b28976d5634ac693430, user_id=a76b340a2b8e46218719b06a20c733e9, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:12.030197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dff78a-e11e-4671-b3f3-8eeeab43dbb4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:12.044035 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1283/2572] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:11 2022] PUT /identity/v3/projects/c18a884cdb3c4b28976d5634ac693430/users/a76b340a2b8e46218719b06a20c733e9/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:12.049208 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b27495-02b8-4150-a670-d393c80d813f None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:12.049521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b27495-02b8-4150-a670-d393c80d813f None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:12.049798 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b27495-02b8-4150-a670-d393c80d813f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:12.070122 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-93b27495-02b8-4150-a670-d393c80d813f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:12.071553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-93b27495-02b8-4150-a670-d393c80d813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:12.131793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1290/2573] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:12.141534 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dff78a-e11e-4671-b3f3-8eeeab43dbb4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:12.143279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-16dff78a-e11e-4671-b3f3-8eeeab43dbb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:12.158956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:12.207158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dff78a-e11e-4671-b3f3-8eeeab43dbb4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:12.211216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fcfe8-44ba-4686-ad94-465f5bb0dd10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:12.211853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fcfe8-44ba-4686-ad94-465f5bb0dd10 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:12.212870 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fcfe8-44ba-4686-ad94-465f5bb0dd10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:12.213046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fcfe8-44ba-4686-ad94-465f5bb0dd10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:12.222533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fcfe8-44ba-4686-ad94-465f5bb0dd10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:12.223991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fcfe8-44ba-4686-ad94-465f5bb0dd10 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:12.235252 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1284/2574] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:12.401975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b27495-02b8-4150-a670-d393c80d813f None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:12.403362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-93b27495-02b8-4150-a670-d393c80d813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:12.468160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:12.502755 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b27495-02b8-4150-a670-d393c80d813f None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:12.509631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a127f8-1de4-46af-a745-d9c150535e1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:12.511326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a127f8-1de4-46af-a745-d9c150535e1b service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:12.511809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a127f8-1de4-46af-a745-d9c150535e1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:12.512400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a127f8-1de4-46af-a745-d9c150535e1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:12.515362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b9a127f8-1de4-46af-a745-d9c150535e1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:12.600071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a127f8-1de4-46af-a745-d9c150535e1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:12.601802 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a127f8-1de4-46af-a745-d9c150535e1b service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:12.612446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1291/2575] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:13.019201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a06d3-8ffb-462a-a983-b479bb55c6be None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:13.019201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a06d3-8ffb-462a-a983-b479bb55c6be None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:13.019767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a06d3-8ffb-462a-a983-b479bb55c6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:13.048705 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-381a06d3-8ffb-462a-a983-b479bb55c6be None None] MFA Rules not processed for user `a76b340a2b8e46218719b06a20c733e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:13.049975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-381a06d3-8ffb-462a-a983-b479bb55c6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:13.136741 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1285/2576] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:13.141984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a127f8-1de4-46af-a745-d9c150535e1b service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:13.155689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a127f8-1de4-46af-a745-d9c150535e1b service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:13.160274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543677e4-96e9-4145-b09a-bc427e956ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:13.160905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543677e4-96e9-4145-b09a-bc427e956ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:13.161184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543677e4-96e9-4145-b09a-bc427e956ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:13.161452 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543677e4-96e9-4145-b09a-bc427e956ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:13.162102 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543677e4-96e9-4145-b09a-bc427e956ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:13.163985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543677e4-96e9-4145-b09a-bc427e956ed7 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:13.200466 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1292/2577] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:13 2022] POST /identity/v3/projects => generated 405 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:40:13.201975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a06d3-8ffb-462a-a983-b479bb55c6be None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:13.215442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a06d3-8ffb-462a-a983-b479bb55c6be None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:13.277780 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28546212-0633-4d66-97db-24005aacbe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:13.278615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28546212-0633-4d66-97db-24005aacbe04 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:13.279039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28546212-0633-4d66-97db-24005aacbe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:13.279313 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28546212-0633-4d66-97db-24005aacbe04 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:13.279960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28546212-0633-4d66-97db-24005aacbe04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:13.282027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28546212-0633-4d66-97db-24005aacbe04 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:13.306744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1286/2578] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:13 2022] POST /identity/v3/users => generated 309 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:13.310534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543677e4-96e9-4145-b09a-bc427e956ed7 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:13.321814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543677e4-96e9-4145-b09a-bc427e956ed7 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:13.326795 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dedc5-5971-40c4-b383-3d77b3bd8822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:13.327487 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dedc5-5971-40c4-b383-3d77b3bd8822 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:13.327830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dedc5-5971-40c4-b383-3d77b3bd8822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:13.328163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dedc5-5971-40c4-b383-3d77b3bd8822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:13.328898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3dedc5-5971-40c4-b383-3d77b3bd8822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:13.330525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3dedc5-5971-40c4-b383-3d77b3bd8822 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:13.341353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1293/2579] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:40:13.345681 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28546212-0633-4d66-97db-24005aacbe04 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:13.356264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28546212-0633-4d66-97db-24005aacbe04 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:13.361114 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168b437-3810-421b-b4f5-400f6ced61a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:13.361763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168b437-3810-421b-b4f5-400f6ced61a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:13.362043 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168b437-3810-421b-b4f5-400f6ced61a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:13.362315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168b437-3810-421b-b4f5-400f6ced61a1 admin admin] PATH_INFO: `/v3/projects/b582034f71284c5fb729cf5c5a02835f/users/87019e5abdf742c4a21b820cf54920e2/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:13.380877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168b437-3810-421b-b4f5-400f6ced61a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b582034f71284c5fb729cf5c5a02835f, user_id=87019e5abdf742c4a21b820cf54920e2, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:13.383526 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168b437-3810-421b-b4f5-400f6ced61a1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:13.398044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1287/2580] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:13 2022] PUT /identity/v3/projects/b582034f71284c5fb729cf5c5a02835f/users/87019e5abdf742c4a21b820cf54920e2/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:13.403037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dedc5-5971-40c4-b383-3d77b3bd8822 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:13.413263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dedc5-5971-40c4-b383-3d77b3bd8822 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:13.473455 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8b528-073a-4c1f-85e7-34d3249d8907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:13.473997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8b528-073a-4c1f-85e7-34d3249d8907 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:13.474296 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8b528-073a-4c1f-85e7-34d3249d8907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:13.474564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8b528-073a-4c1f-85e7-34d3249d8907 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:13.475218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8b528-073a-4c1f-85e7-34d3249d8907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:13.476803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8b528-073a-4c1f-85e7-34d3249d8907 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:13.487165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1294/2581] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:40:13.491525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168b437-3810-421b-b4f5-400f6ced61a1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:13.502324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168b437-3810-421b-b4f5-400f6ced61a1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:13.506839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8810ab1-a326-4660-a647-b6406bf1115a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:13.507477 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8810ab1-a326-4660-a647-b6406bf1115a admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:13.507753 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8810ab1-a326-4660-a647-b6406bf1115a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:13.508022 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8810ab1-a326-4660-a647-b6406bf1115a admin admin] PATH_INFO: `/v3/projects/b582034f71284c5fb729cf5c5a02835f/users/87019e5abdf742c4a21b820cf54920e2/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:13.509984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8810ab1-a326-4660-a647-b6406bf1115a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b582034f71284c5fb729cf5c5a02835f, user_id=87019e5abdf742c4a21b820cf54920e2, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:13.512582 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8810ab1-a326-4660-a647-b6406bf1115a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:13.518027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:13.523418 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1288/2582] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:13 2022] PUT /identity/v3/projects/b582034f71284c5fb729cf5c5a02835f/users/87019e5abdf742c4a21b820cf54920e2/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:14.459696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ec1fb-1791-48b6-a042-0762ba9fa5ee None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:14.460237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ec1fb-1791-48b6-a042-0762ba9fa5ee None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:14.460237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ec1fb-1791-48b6-a042-0762ba9fa5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:14.490270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-461ec1fb-1791-48b6-a042-0762ba9fa5ee None None] MFA Rules not processed for user `87019e5abdf742c4a21b820cf54920e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:14.491734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-461ec1fb-1791-48b6-a042-0762ba9fa5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:14.538269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:14.583826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1295/2583] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:14 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:14.945516 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8810ab1-a326-4660-a647-b6406bf1115a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:14.948164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e8810ab1-a326-4660-a647-b6406bf1115a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:15.028373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8810ab1-a326-4660-a647-b6406bf1115a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:15.033655 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:15.034290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:15.034566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:15.034838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] PATH_INFO: `/v3/users/ac2ea6ec3fe640e182fa689af0ec6f3c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:15.035956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac2ea6ec3fe640e182fa689af0ec6f3c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:15.038985 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:15.120159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac2ea6ec3fe640e182fa689af0ec6f3c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:15.120774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac2ea6ec3fe640e182fa689af0ec6f3c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:15.121415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac2ea6ec3fe640e182fa689af0ec6f3c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:15.137327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac2ea6ec3fe640e182fa689af0ec6f3c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:15.145354 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac2ea6ec3fe640e182fa689af0ec6f3c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:15.168856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1289/2584] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:14 2022] DELETE /identity/v3/users/ac2ea6ec3fe640e182fa689af0ec6f3c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:15.170490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566b633-33fa-41b7-807c-9832d963266a None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:15.170781 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566b633-33fa-41b7-807c-9832d963266a None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:15.171072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566b633-33fa-41b7-807c-9832d963266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:15.202090 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-5566b633-33fa-41b7-807c-9832d963266a None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:15.204127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5566b633-33fa-41b7-807c-9832d963266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:15.291950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1296/2585] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:15.309623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:15.312161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:15.421004 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2330ae2-ea3d-4776-a405-3bb5c81e5e6e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:15.425627 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db20b17-27ec-44bb-922d-6277fea3222d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:15.426379 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db20b17-27ec-44bb-922d-6277fea3222d service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:15.426663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db20b17-27ec-44bb-922d-6277fea3222d service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:15.426943 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db20b17-27ec-44bb-922d-6277fea3222d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:15.441506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db20b17-27ec-44bb-922d-6277fea3222d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:15.442979 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db20b17-27ec-44bb-922d-6277fea3222d service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:15.455248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1290/2586] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:16.033109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566b633-33fa-41b7-807c-9832d963266a None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:16.034616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5566b633-33fa-41b7-807c-9832d963266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:16.047041 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566b633-33fa-41b7-807c-9832d963266a None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:16.059390 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:16.060062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:16.060541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:16.060834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] PATH_INFO: `/v3/projects/846c48df6b6045a1b1215bcfb3ebb637` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:16.062177 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=846c48df6b6045a1b1215bcfb3ebb637)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:16.064510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:16.131054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846c48df6b6045a1b1215bcfb3ebb637', 'request_id': 'req-08a3b0f8-640c-49e7-9d8e-8b1592badb04'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.146661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846c48df6b6045a1b1215bcfb3ebb637', 'request_id': 'req-08a3b0f8-640c-49e7-9d8e-8b1592badb04'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.156976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846c48df6b6045a1b1215bcfb3ebb637', 'request_id': 'req-08a3b0f8-640c-49e7-9d8e-8b1592badb04'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.193070 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] The token cache is being invalidate because project 846c48df6b6045a1b1215bcfb3ebb637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:16.193213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.193895 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.195459 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1297/2587] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:16 2022] DELETE /identity/v3/projects/846c48df6b6045a1b1215bcfb3ebb637 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:16.200803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db20b17-27ec-44bb-922d-6277fea3222d service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:16.231838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db20b17-27ec-44bb-922d-6277fea3222d service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:16.244986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:16.246479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:16.262724 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:16.313656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:16.314436 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:16.314720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:16.314993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] PATH_INFO: `/v3/users/ef166afde72a4222b858f474d66ecf9a` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:16.316130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef166afde72a4222b858f474d66ecf9a)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:16.318086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:16.327516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:16.336194 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3b0f8-640c-49e7-9d8e-8b1592badb04 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:16.345385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:16.346634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:16.346936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:16.347207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] PATH_INFO: `/v3/users/a76b340a2b8e46218719b06a20c733e9` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:16.348477 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76b340a2b8e46218719b06a20c733e9)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:16.350486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:16.375752 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef166afde72a4222b858f474d66ecf9a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.376634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef166afde72a4222b858f474d66ecf9a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.377437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef166afde72a4222b858f474d66ecf9a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.394389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef166afde72a4222b858f474d66ecf9a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.401465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76b340a2b8e46218719b06a20c733e9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.402204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76b340a2b8e46218719b06a20c733e9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.403023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76b340a2b8e46218719b06a20c733e9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.405005 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef166afde72a4222b858f474d66ecf9a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.411294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76b340a2b8e46218719b06a20c733e9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.422106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1291/2588] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:16 2022] DELETE /identity/v3/users/ef166afde72a4222b858f474d66ecf9a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:16.425616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76b340a2b8e46218719b06a20c733e9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:16.440940 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1298/2589] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:16 2022] DELETE /identity/v3/users/a76b340a2b8e46218719b06a20c733e9 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:16.705727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:16.707314 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:16.833733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea803b-6bb3-4097-a0b7-0622a9f79bc4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:16.855117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:16.857139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:16.911319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:16.936205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:16.936910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:16.937213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:16.937613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] PATH_INFO: `/v3/projects/c18a884cdb3c4b28976d5634ac693430` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:16.940383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c18a884cdb3c4b28976d5634ac693430)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:16.942328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:16.963135 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638aee6-8064-4bf3-880d-b18276d507e3 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:16.968151 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:16.969105 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:16.969449 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:16.969788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] PATH_INFO: `/v3/projects/c5c73e49194b40c49f89eac4560a4286` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:16.971109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5c73e49194b40c49f89eac4560a4286)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:16.973168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:17.006419 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c18a884cdb3c4b28976d5634ac693430', 'request_id': 'req-dd6fe61c-9529-4438-84f7-b5e693e33ef3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.012974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c18a884cdb3c4b28976d5634ac693430', 'request_id': 'req-dd6fe61c-9529-4438-84f7-b5e693e33ef3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.022399 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c18a884cdb3c4b28976d5634ac693430', 'request_id': 'req-dd6fe61c-9529-4438-84f7-b5e693e33ef3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.037374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5c73e49194b40c49f89eac4560a4286', 'request_id': 'req-778aa179-2290-4db9-8610-431eb443951c'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.040541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] The token cache is being invalidate because project c18a884cdb3c4b28976d5634ac693430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:17.040992 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.041707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.043435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1292/2590] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:16 2022] DELETE /identity/v3/projects/c18a884cdb3c4b28976d5634ac693430 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:17.044944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5c73e49194b40c49f89eac4560a4286', 'request_id': 'req-778aa179-2290-4db9-8610-431eb443951c'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.048549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:17.049972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:17.050981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5c73e49194b40c49f89eac4560a4286', 'request_id': 'req-778aa179-2290-4db9-8610-431eb443951c'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.066222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] The token cache is being invalidate because project c5c73e49194b40c49f89eac4560a4286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:17.066514 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.067126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.068631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1299/2591] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:16 2022] DELETE /identity/v3/projects/c5c73e49194b40c49f89eac4560a4286 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:17.073032 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:17.074509 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:17.081160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:17.122509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fe61c-9529-4438-84f7-b5e693e33ef3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:17.142435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778aa179-2290-4db9-8610-431eb443951c admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:17.147148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:17.147810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.148089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.148364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] PATH_INFO: `/v3/users/8fdddd41b1db4a21a32f77f678bbebdb` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.149828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fdddd41b1db4a21a32f77f678bbebdb)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:17.151793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:17.180361 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:17.182641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.183274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.183614 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] PATH_INFO: `/v3/users/87019e5abdf742c4a21b820cf54920e2` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.185006 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=87019e5abdf742c4a21b820cf54920e2)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:17.187146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:17.201107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fdddd41b1db4a21a32f77f678bbebdb'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.202028 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fdddd41b1db4a21a32f77f678bbebdb'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.202640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fdddd41b1db4a21a32f77f678bbebdb'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.210585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fdddd41b1db4a21a32f77f678bbebdb'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.215451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:17.224529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fdddd41b1db4a21a32f77f678bbebdb'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.236869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87019e5abdf742c4a21b820cf54920e2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.237607 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87019e5abdf742c4a21b820cf54920e2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.238141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1300/2592] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:17 2022] DELETE /identity/v3/users/8fdddd41b1db4a21a32f77f678bbebdb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:17.241175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87019e5abdf742c4a21b820cf54920e2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.254751 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87019e5abdf742c4a21b820cf54920e2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.262134 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87019e5abdf742c4a21b820cf54920e2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.276426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1293/2593] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:17 2022] DELETE /identity/v3/users/87019e5abdf742c4a21b820cf54920e2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:17.464997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:17.466352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:17.525345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:17.527522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:17.543432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4e1b8-7c51-47f4-93d1-5ce4ec7b54cd admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:17.573849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:17.606022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb2273-f5a4-430f-80e1-2a127fbed3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:17.606668 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb2273-f5a4-430f-80e1-2a127fbed3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.606947 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb2273-f5a4-430f-80e1-2a127fbed3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.607219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb2273-f5a4-430f-80e1-2a127fbed3ff admin admin] PATH_INFO: `/v3/projects/b582034f71284c5fb729cf5c5a02835f` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.608372 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb2273-f5a4-430f-80e1-2a127fbed3ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=b582034f71284c5fb729cf5c5a02835f)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:17.609709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4e4b-ab09-4713-82f9-186955e957dc admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:17.610332 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb2273-f5a4-430f-80e1-2a127fbed3ff admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:17.614243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304678d4-286f-42c4-b54e-1d1d60c30342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:17.614866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304678d4-286f-42c4-b54e-1d1d60c30342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.615144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304678d4-286f-42c4-b54e-1d1d60c30342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.615410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304678d4-286f-42c4-b54e-1d1d60c30342 admin admin] PATH_INFO: `/v3/projects/2c431786e54f4a4d8427fe183db8c443` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.616969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304678d4-286f-42c4-b54e-1d1d60c30342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c431786e54f4a4d8427fe183db8c443)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:17.619098 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304678d4-286f-42c4-b54e-1d1d60c30342 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:17.664628 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-94bb2273-f5a4-430f-80e1-2a127fbed3ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b582034f71284c5fb729cf5c5a02835f', 'request_id': 'req-94bb2273-f5a4-430f-80e1-2a127fbed3ff'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.668106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-304678d4-286f-42c4-b54e-1d1d60c30342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c431786e54f4a4d8427fe183db8c443', 'request_id': 'req-304678d4-286f-42c4-b54e-1d1d60c30342'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.671498 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-94bb2273-f5a4-430f-80e1-2a127fbed3ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b582034f71284c5fb729cf5c5a02835f', 'request_id': 'req-94bb2273-f5a4-430f-80e1-2a127fbed3ff'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.674151 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-304678d4-286f-42c4-b54e-1d1d60c30342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c431786e54f4a4d8427fe183db8c443', 'request_id': 'req-304678d4-286f-42c4-b54e-1d1d60c30342'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.677183 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-94bb2273-f5a4-430f-80e1-2a127fbed3ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b582034f71284c5fb729cf5c5a02835f', 'request_id': 'req-94bb2273-f5a4-430f-80e1-2a127fbed3ff'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.681284 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-304678d4-286f-42c4-b54e-1d1d60c30342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c431786e54f4a4d8427fe183db8c443', 'request_id': 'req-304678d4-286f-42c4-b54e-1d1d60c30342'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.691460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-94bb2273-f5a4-430f-80e1-2a127fbed3ff admin admin] The token cache is being invalidate because project b582034f71284c5fb729cf5c5a02835f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:17.691746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-94bb2273-f5a4-430f-80e1-2a127fbed3ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.692364 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-94bb2273-f5a4-430f-80e1-2a127fbed3ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.694138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1301/2594] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:17 2022] DELETE /identity/v3/projects/b582034f71284c5fb729cf5c5a02835f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:17.694269 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-304678d4-286f-42c4-b54e-1d1d60c30342 admin admin] The token cache is being invalidate because project 2c431786e54f4a4d8427fe183db8c443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:17.694478 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-304678d4-286f-42c4-b54e-1d1d60c30342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.695119 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-304678d4-286f-42c4-b54e-1d1d60c30342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:17.697271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1294/2595] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:17 2022] DELETE /identity/v3/projects/2c431786e54f4a4d8427fe183db8c443 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:17.702857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abcf07-27ce-41f2-ad43-8d500dfd5507 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.702985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a05f3-c6c5-4d2e-9528-db16a9b7ebc6 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.703140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abcf07-27ce-41f2-ad43-8d500dfd5507 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.703260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a05f3-c6c5-4d2e-9528-db16a9b7ebc6 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.703413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abcf07-27ce-41f2-ad43-8d500dfd5507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.703523 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a05f3-c6c5-4d2e-9528-db16a9b7ebc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.722923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-90abcf07-27ce-41f2-ad43-8d500dfd5507 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:17.723156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d52a05f3-c6c5-4d2e-9528-db16a9b7ebc6 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:17.724155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-90abcf07-27ce-41f2-ad43-8d500dfd5507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:17.724770 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d52a05f3-c6c5-4d2e-9528-db16a9b7ebc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:17.765258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:17.793223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1302/2596] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:17.794513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424cca4-088e-44eb-8ed8-096b55435e78 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.794802 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424cca4-088e-44eb-8ed8-096b55435e78 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.795080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424cca4-088e-44eb-8ed8-096b55435e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.805746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1295/2597] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:17.807754 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7047cf-5d2d-40f2-8b63-c957c835ba23 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.808046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7047cf-5d2d-40f2-8b63-c957c835ba23 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.808321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7047cf-5d2d-40f2-8b63-c957c835ba23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.817211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-7424cca4-088e-44eb-8ed8-096b55435e78 None None] MFA Rules not processed for user `f989b6fded53458b85b39789a32a39e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:17.818531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7424cca4-088e-44eb-8ed8-096b55435e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:17.829183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0b7047cf-5d2d-40f2-8b63-c957c835ba23 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:17.830372 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0b7047cf-5d2d-40f2-8b63-c957c835ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:17.839522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1296/2598] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:17.840700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa321053-9909-454d-9630-a8ff41cd33dc None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.841018 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa321053-9909-454d-9630-a8ff41cd33dc None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.841291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa321053-9909-454d-9630-a8ff41cd33dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.862321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-fa321053-9909-454d-9630-a8ff41cd33dc None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:17.863592 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fa321053-9909-454d-9630-a8ff41cd33dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:17.874233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1297/2599] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:17.875040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa321053-9909-454d-9630-a8ff41cd33dc None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:17.877226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1303/2600] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:17.881666 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424cca4-088e-44eb-8ed8-096b55435e78 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:17.897262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa321053-9909-454d-9630-a8ff41cd33dc None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:17.897262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb701bb-af68-4a92-8e11-269775882b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:17.897262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb701bb-af68-4a92-8e11-269775882b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.897262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb701bb-af68-4a92-8e11-269775882b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.897595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb701bb-af68-4a92-8e11-269775882b7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.897595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb701bb-af68-4a92-8e11-269775882b7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:40:17.897595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb701bb-af68-4a92-8e11-269775882b7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:17.897595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb701bb-af68-4a92-8e11-269775882b7f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:17.897871 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424cca4-088e-44eb-8ed8-096b55435e78 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:17.909941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4f8df-958c-4a4c-a93a-e464f6e919ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:17.913651 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1298/2601] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:40:17.913797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4f8df-958c-4a4c-a93a-e464f6e919ac admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.913797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4f8df-958c-4a4c-a93a-e464f6e919ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.913797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4f8df-958c-4a4c-a93a-e464f6e919ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.913797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4f8df-958c-4a4c-a93a-e464f6e919ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:40:17.913797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4f8df-958c-4a4c-a93a-e464f6e919ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:17.914155 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4f8df-958c-4a4c-a93a-e464f6e919ac admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:17.914802 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb701bb-af68-4a92-8e11-269775882b7f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:17.916324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:17.926693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb701bb-af68-4a92-8e11-269775882b7f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:17.936803 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b94ef9-95a4-44fb-84a5-11a302a19b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:17.936993 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1304/2602] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:40:17.937093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b94ef9-95a4-44fb-84a5-11a302a19b9f admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.937093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b94ef9-95a4-44fb-84a5-11a302a19b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.937093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b94ef9-95a4-44fb-84a5-11a302a19b9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.938262 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b94ef9-95a4-44fb-84a5-11a302a19b9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:17.939661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4f8df-958c-4a4c-a93a-e464f6e919ac admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:17.939789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b94ef9-95a4-44fb-84a5-11a302a19b9f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:17.950942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4f8df-958c-4a4c-a93a-e464f6e919ac admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:17.956186 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f53c8-3781-45d6-b4dc-06dd6733659e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:17.956893 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f53c8-3781-45d6-b4dc-06dd6733659e admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:17.957252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f53c8-3781-45d6-b4dc-06dd6733659e admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:17.957605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f53c8-3781-45d6-b4dc-06dd6733659e admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:17.958426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f53c8-3781-45d6-b4dc-06dd6733659e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:17.961184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f53c8-3781-45d6-b4dc-06dd6733659e admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:17.979312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1299/2603] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:17 2022] POST /identity/v3/projects => generated 391 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:17.984040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b94ef9-95a4-44fb-84a5-11a302a19b9f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.023817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1305/2604] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:17 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:40:18.026264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f53c8-3781-45d6-b4dc-06dd6733659e admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.075429 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b94ef9-95a4-44fb-84a5-11a302a19b9f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.140255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f53c8-3781-45d6-b4dc-06dd6733659e admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.153015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbc034-dae5-4eaa-a96b-523eb0cec9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.153994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbc034-dae5-4eaa-a96b-523eb0cec9d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.154584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbc034-dae5-4eaa-a96b-523eb0cec9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.155024 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbc034-dae5-4eaa-a96b-523eb0cec9d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.156366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbc034-dae5-4eaa-a96b-523eb0cec9d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.158965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbc034-dae5-4eaa-a96b-523eb0cec9d8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.165419 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb672e7-6532-4126-8aa2-415a72389116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.166428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb672e7-6532-4126-8aa2-415a72389116 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.166782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb672e7-6532-4126-8aa2-415a72389116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.167130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb672e7-6532-4126-8aa2-415a72389116 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.167829 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb672e7-6532-4126-8aa2-415a72389116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.170874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb672e7-6532-4126-8aa2-415a72389116 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.195837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1306/2605] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:18 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:18.201353 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbc034-dae5-4eaa-a96b-523eb0cec9d8 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.203597 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1300/2606] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:17 2022] POST /identity/v3/users => generated 296 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:18.208118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb672e7-6532-4126-8aa2-415a72389116 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.215020 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbc034-dae5-4eaa-a96b-523eb0cec9d8 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.221385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d86972-982a-416b-a975-4bd56d48ffde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.222221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d86972-982a-416b-a975-4bd56d48ffde admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.222701 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb672e7-6532-4126-8aa2-415a72389116 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.223335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d86972-982a-416b-a975-4bd56d48ffde admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.223815 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d86972-982a-416b-a975-4bd56d48ffde admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.224653 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d86972-982a-416b-a975-4bd56d48ffde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.226467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d86972-982a-416b-a975-4bd56d48ffde admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.228875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b5572-b963-49b2-a1cc-e12fadf8161b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.230071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b5572-b963-49b2-a1cc-e12fadf8161b admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.231282 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b5572-b963-49b2-a1cc-e12fadf8161b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.231946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b5572-b963-49b2-a1cc-e12fadf8161b admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.232690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b5572-b963-49b2-a1cc-e12fadf8161b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.234493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b5572-b963-49b2-a1cc-e12fadf8161b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.241571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1307/2607] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:18.245297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d86972-982a-416b-a975-4bd56d48ffde admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.249239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1301/2608] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:18.254671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b5572-b963-49b2-a1cc-e12fadf8161b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.258245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d86972-982a-416b-a975-4bd56d48ffde admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.266636 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b5572-b963-49b2-a1cc-e12fadf8161b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.267434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4c011-4158-494b-a0c1-ca9a96c96877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.268199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4c011-4158-494b-a0c1-ca9a96c96877 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.268509 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4c011-4158-494b-a0c1-ca9a96c96877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.268982 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4c011-4158-494b-a0c1-ca9a96c96877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.269876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4c011-4158-494b-a0c1-ca9a96c96877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.271557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4c011-4158-494b-a0c1-ca9a96c96877 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.284431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc287d53-f253-4ead-bad6-9a1f08dd0559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.285334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc287d53-f253-4ead-bad6-9a1f08dd0559 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.285794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc287d53-f253-4ead-bad6-9a1f08dd0559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.286239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc287d53-f253-4ead-bad6-9a1f08dd0559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.286374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1308/2609] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:18.287173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc287d53-f253-4ead-bad6-9a1f08dd0559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.289077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc287d53-f253-4ead-bad6-9a1f08dd0559 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.291213 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4c011-4158-494b-a0c1-ca9a96c96877 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.299644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:18.304221 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4c011-4158-494b-a0c1-ca9a96c96877 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.308429 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1302/2610] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:40:18.308857 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc287d53-f253-4ead-bad6-9a1f08dd0559 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.311787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bc287d53-f253-4ead-bad6-9a1f08dd0559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:18.314047 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8a3b0-db6c-4038-964f-3932677a0541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.314769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8a3b0-db6c-4038-964f-3932677a0541 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.315159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8a3b0-db6c-4038-964f-3932677a0541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.315433 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8a3b0-db6c-4038-964f-3932677a0541 admin admin] PATH_INFO: `/v3/projects/33a96d7c465f422aa6026281c3267da3/users/152710e047354fbbb4aa5a0945e951e4/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.331924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8a3b0-db6c-4038-964f-3932677a0541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a96d7c465f422aa6026281c3267da3, user_id=152710e047354fbbb4aa5a0945e951e4, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.334697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8a3b0-db6c-4038-964f-3932677a0541 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.349072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1309/2611] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:18 2022] PUT /identity/v3/projects/33a96d7c465f422aa6026281c3267da3/users/152710e047354fbbb4aa5a0945e951e4/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:18.349968 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8a3b0-db6c-4038-964f-3932677a0541 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.378661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc287d53-f253-4ead-bad6-9a1f08dd0559 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.419145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8a3b0-db6c-4038-964f-3932677a0541 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.423872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149fadb-cdf7-4f4d-bbb9-c707cd497d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.424525 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149fadb-cdf7-4f4d-bbb9-c707cd497d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.424839 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149fadb-cdf7-4f4d-bbb9-c707cd497d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.425118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149fadb-cdf7-4f4d-bbb9-c707cd497d70 admin admin] PATH_INFO: `/v3/projects/3c99be3b3af24f97a9fc25c9ea6f06ef/users/feaeaf2d6c084c098baaecad6490ade3/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.428178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3c516-4d78-4cbc-a355-64b0ea4250f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.428834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3c516-4d78-4cbc-a355-64b0ea4250f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.429083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3c516-4d78-4cbc-a355-64b0ea4250f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.429353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3c516-4d78-4cbc-a355-64b0ea4250f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.431248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a7b3c516-4d78-4cbc-a355-64b0ea4250f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:18.441010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149fadb-cdf7-4f4d-bbb9-c707cd497d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c99be3b3af24f97a9fc25c9ea6f06ef, user_id=feaeaf2d6c084c098baaecad6490ade3, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.441536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3c516-4d78-4cbc-a355-64b0ea4250f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.443152 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3c516-4d78-4cbc-a355-64b0ea4250f9 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.444031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149fadb-cdf7-4f4d-bbb9-c707cd497d70 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.452914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1303/2612] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:18.454184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba30c8-d5f5-4be4-b027-a97234658767 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.454475 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba30c8-d5f5-4be4-b027-a97234658767 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.454818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba30c8-d5f5-4be4-b027-a97234658767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.461577 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:18.479150 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1310/2613] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:18 2022] PUT /identity/v3/projects/3c99be3b3af24f97a9fc25c9ea6f06ef/users/feaeaf2d6c084c098baaecad6490ade3/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:18.481678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644163b-d573-4e20-a0d8-f832ddc8b5d1 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.482009 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644163b-d573-4e20-a0d8-f832ddc8b5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.482241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644163b-d573-4e20-a0d8-f832ddc8b5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.499084 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-aaba30c8-d5f5-4be4-b027-a97234658767 None None] MFA Rules not processed for user `152710e047354fbbb4aa5a0945e951e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:18.500401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-aaba30c8-d5f5-4be4-b027-a97234658767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:18.512392 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f644163b-d573-4e20-a0d8-f832ddc8b5d1 None None] MFA Rules not processed for user `feaeaf2d6c084c098baaecad6490ade3`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:18.514022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f644163b-d573-4e20-a0d8-f832ddc8b5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:18.587688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1304/2614] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:18.592101 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba30c8-d5f5-4be4-b027-a97234658767 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.606748 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1311/2615] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:40:18.607824 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644163b-d573-4e20-a0d8-f832ddc8b5d1 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.661256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:18.678787 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba30c8-d5f5-4be4-b027-a97234658767 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.686833 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c571c0f-6f5b-4795-a595-e73dc6f9dc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.687606 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c571c0f-6f5b-4795-a595-e73dc6f9dc87 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.688208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c571c0f-6f5b-4795-a595-e73dc6f9dc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.688550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c571c0f-6f5b-4795-a595-e73dc6f9dc87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.689290 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c571c0f-6f5b-4795-a595-e73dc6f9dc87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.691278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c571c0f-6f5b-4795-a595-e73dc6f9dc87 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.698278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644163b-d573-4e20-a0d8-f832ddc8b5d1 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.703173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badc62a-1a3e-4f46-95cd-1cd49affa86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.703723 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badc62a-1a3e-4f46-95cd-1cd49affa86f admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.704512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badc62a-1a3e-4f46-95cd-1cd49affa86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.704512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badc62a-1a3e-4f46-95cd-1cd49affa86f admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.705106 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badc62a-1a3e-4f46-95cd-1cd49affa86f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.707160 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badc62a-1a3e-4f46-95cd-1cd49affa86f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.727634 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1305/2616] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:18 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:18.731637 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c571c0f-6f5b-4795-a595-e73dc6f9dc87 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.746313 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1312/2617] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:18 2022] POST /identity/v3/projects => generated 391 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:18.747407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badc62a-1a3e-4f46-95cd-1cd49affa86f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.804157 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c571c0f-6f5b-4795-a595-e73dc6f9dc87 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.824767 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badc62a-1a3e-4f46-95cd-1cd49affa86f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.830271 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a0d00-5d2b-49da-b205-4891f65729c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.830914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a0d00-5d2b-49da-b205-4891f65729c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.831201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a0d00-5d2b-49da-b205-4891f65729c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.831479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a0d00-5d2b-49da-b205-4891f65729c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.832110 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a0d00-5d2b-49da-b205-4891f65729c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.834140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a0d00-5d2b-49da-b205-4891f65729c3 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.861790 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1313/2618] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:18 2022] POST /identity/v3/users => generated 302 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:18.862873 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a0d00-5d2b-49da-b205-4891f65729c3 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.869218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe966e-82ba-422a-a3f2-db76081ed28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.870006 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe966e-82ba-422a-a3f2-db76081ed28b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.870288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe966e-82ba-422a-a3f2-db76081ed28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.870630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe966e-82ba-422a-a3f2-db76081ed28b admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.871328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe966e-82ba-422a-a3f2-db76081ed28b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.871846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:18.873430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe966e-82ba-422a-a3f2-db76081ed28b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.875768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a0d00-5d2b-49da-b205-4891f65729c3 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.885290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138819f0-373b-4624-b866-ee83a0302ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.885995 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138819f0-373b-4624-b866-ee83a0302ead admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.887267 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138819f0-373b-4624-b866-ee83a0302ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.887552 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138819f0-373b-4624-b866-ee83a0302ead admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.888228 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138819f0-373b-4624-b866-ee83a0302ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.890259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138819f0-373b-4624-b866-ee83a0302ead admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.901074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1306/2619] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:18 2022] POST /identity/v3/users => generated 295 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:18.904988 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe966e-82ba-422a-a3f2-db76081ed28b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.905696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1314/2620] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:18.909532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138819f0-373b-4624-b866-ee83a0302ead admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.920710 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138819f0-373b-4624-b866-ee83a0302ead admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.921112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe966e-82ba-422a-a3f2-db76081ed28b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.925674 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5fa9f-02e8-40c1-8a25-fe329f91e8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.926335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5fa9f-02e8-40c1-8a25-fe329f91e8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.926451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292ca93-6446-4514-932d-a7b88bcc34a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:18.926624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5fa9f-02e8-40c1-8a25-fe329f91e8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.926902 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5fa9f-02e8-40c1-8a25-fe329f91e8bd admin admin] PATH_INFO: `/v3/projects/a988fa00f32742bfa2083c58b007c21b/users/5fca198dd89b46d49fbe64783e95479c/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.927201 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292ca93-6446-4514-932d-a7b88bcc34a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:18.927332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292ca93-6446-4514-932d-a7b88bcc34a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:18.927616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292ca93-6446-4514-932d-a7b88bcc34a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:18.928442 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292ca93-6446-4514-932d-a7b88bcc34a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.931515 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292ca93-6446-4514-932d-a7b88bcc34a2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.945563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1307/2621] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:40:18.946481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5fa9f-02e8-40c1-8a25-fe329f91e8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a988fa00f32742bfa2083c58b007c21b, user_id=5fca198dd89b46d49fbe64783e95479c, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:18.949515 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5fa9f-02e8-40c1-8a25-fe329f91e8bd admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:18.949772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292ca93-6446-4514-932d-a7b88bcc34a2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:18.965063 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292ca93-6446-4514-932d-a7b88bcc34a2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:18.975264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1315/2622] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:18 2022] PUT /identity/v3/projects/a988fa00f32742bfa2083c58b007c21b/users/5fca198dd89b46d49fbe64783e95479c/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:18.980252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5fa9f-02e8-40c1-8a25-fe329f91e8bd admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:19.054258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3f640-fe2e-4854-ac1b-731d6ba10861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:19.054930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3f640-fe2e-4854-ac1b-731d6ba10861 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.055240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3f640-fe2e-4854-ac1b-731d6ba10861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.055522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3f640-fe2e-4854-ac1b-731d6ba10861 admin admin] PATH_INFO: `/v3/projects/e6ee99f9409e44f0bf1484ac8f6197f2/users/dded912f2b664f09b9f8d4a735ebb354/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.058496 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:19.066176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5fa9f-02e8-40c1-8a25-fe329f91e8bd admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:19.073912 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a7aaa-7b19-46ce-a897-ecf916a91631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:19.074684 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a7aaa-7b19-46ce-a897-ecf916a91631 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.075138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a7aaa-7b19-46ce-a897-ecf916a91631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.075584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a7aaa-7b19-46ce-a897-ecf916a91631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.076323 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3f640-fe2e-4854-ac1b-731d6ba10861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6ee99f9409e44f0bf1484ac8f6197f2, user_id=dded912f2b664f09b9f8d4a735ebb354, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:19.076458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a7aaa-7b19-46ce-a897-ecf916a91631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:19.078115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a7aaa-7b19-46ce-a897-ecf916a91631 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:19.079446 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3f640-fe2e-4854-ac1b-731d6ba10861 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:19.090358 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1316/2623] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:19.094829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a7aaa-7b19-46ce-a897-ecf916a91631 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:19.099918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1308/2624] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:18 2022] PUT /identity/v3/projects/e6ee99f9409e44f0bf1484ac8f6197f2/users/dded912f2b664f09b9f8d4a735ebb354/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:19.104610 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3f640-fe2e-4854-ac1b-731d6ba10861 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:19.109218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a7aaa-7b19-46ce-a897-ecf916a91631 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:19.173758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae228fe3-bf3e-49b5-9d90-e403f7ce4353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:19.174495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae228fe3-bf3e-49b5-9d90-e403f7ce4353 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.174681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae228fe3-bf3e-49b5-9d90-e403f7ce4353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.174958 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae228fe3-bf3e-49b5-9d90-e403f7ce4353 admin admin] PATH_INFO: `/v3/projects/a988fa00f32742bfa2083c58b007c21b/users/5fca198dd89b46d49fbe64783e95479c/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.178083 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae228fe3-bf3e-49b5-9d90-e403f7ce4353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a988fa00f32742bfa2083c58b007c21b, user_id=5fca198dd89b46d49fbe64783e95479c, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:19.179156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3f640-fe2e-4854-ac1b-731d6ba10861 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:19.181066 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae228fe3-bf3e-49b5-9d90-e403f7ce4353 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:19.184514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517baf6-6960-4a5b-91a3-5e7d8424a362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:19.185102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517baf6-6960-4a5b-91a3-5e7d8424a362 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.185429 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517baf6-6960-4a5b-91a3-5e7d8424a362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.185658 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517baf6-6960-4a5b-91a3-5e7d8424a362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.186368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517baf6-6960-4a5b-91a3-5e7d8424a362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:19.188286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517baf6-6960-4a5b-91a3-5e7d8424a362 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:19.195323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1317/2625] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:19 2022] PUT /identity/v3/projects/a988fa00f32742bfa2083c58b007c21b/users/5fca198dd89b46d49fbe64783e95479c/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:19.203678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1309/2626] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:40:19.206211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae228fe3-bf3e-49b5-9d90-e403f7ce4353 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:19.207068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29891b-ba08-473c-8ead-cfe665872796 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.207355 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29891b-ba08-473c-8ead-cfe665872796 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.207668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29891b-ba08-473c-8ead-cfe665872796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.242789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6a29891b-ba08-473c-8ead-cfe665872796 None None] MFA Rules not processed for user `5fca198dd89b46d49fbe64783e95479c`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:19.244238 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6a29891b-ba08-473c-8ead-cfe665872796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:19.273029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:19.278726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae228fe3-bf3e-49b5-9d90-e403f7ce4353 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:19.286973 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648896e9-eb3f-4623-8982-38a599751679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:19.287643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648896e9-eb3f-4623-8982-38a599751679 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.287927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648896e9-eb3f-4623-8982-38a599751679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.288205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648896e9-eb3f-4623-8982-38a599751679 admin admin] PATH_INFO: `/v3/projects/e6ee99f9409e44f0bf1484ac8f6197f2/users/dded912f2b664f09b9f8d4a735ebb354/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.290634 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648896e9-eb3f-4623-8982-38a599751679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6ee99f9409e44f0bf1484ac8f6197f2, user_id=dded912f2b664f09b9f8d4a735ebb354, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:19.293372 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648896e9-eb3f-4623-8982-38a599751679 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:19.305635 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1318/2627] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:19 2022] PUT /identity/v3/projects/e6ee99f9409e44f0bf1484ac8f6197f2/users/dded912f2b664f09b9f8d4a735ebb354/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:19.317508 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d3d7c-9831-49ce-ad64-2e6c027c7930 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.317805 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d3d7c-9831-49ce-ad64-2e6c027c7930 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.318068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d3d7c-9831-49ce-ad64-2e6c027c7930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.343707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1310/2628] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:19.355617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29891b-ba08-473c-8ead-cfe665872796 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:19.357146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-9a7d3d7c-9831-49ce-ad64-2e6c027c7930 None None] MFA Rules not processed for user `dded912f2b664f09b9f8d4a735ebb354`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:19.357708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6a29891b-ba08-473c-8ead-cfe665872796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:19.358350 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9a7d3d7c-9831-49ce-ad64-2e6c027c7930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:19.428995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29891b-ba08-473c-8ead-cfe665872796 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:19.434407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1319/2629] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:19.435404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d3d7c-9831-49ce-ad64-2e6c027c7930 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:19.436971 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9a7d3d7c-9831-49ce-ad64-2e6c027c7930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:19.450462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:19.497842 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937cc3d-919d-461a-a12a-d06679373232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:19.498467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937cc3d-919d-461a-a12a-d06679373232 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.498877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937cc3d-919d-461a-a12a-d06679373232 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.499776 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937cc3d-919d-461a-a12a-d06679373232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.519473 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d3d7c-9831-49ce-ad64-2e6c027c7930 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:19.524046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:19.525205 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.525480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.525747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] PATH_INFO: `/v3/users/c4b395961baa4e6dbcefa46e45145a45` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.526979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4b395961baa4e6dbcefa46e45145a45)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:19.528914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:19.566053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937cc3d-919d-461a-a12a-d06679373232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:19.567533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937cc3d-919d-461a-a12a-d06679373232 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:19.577903 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1311/2630] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:19 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:40:19.578604 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4b395961baa4e6dbcefa46e45145a45'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:19.578752 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937cc3d-919d-461a-a12a-d06679373232 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:19.579343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4b395961baa4e6dbcefa46e45145a45'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:19.579969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b937cc3d-919d-461a-a12a-d06679373232 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:19.580099 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4b395961baa4e6dbcefa46e45145a45'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:19.589635 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4b395961baa4e6dbcefa46e45145a45'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:19.591087 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937cc3d-919d-461a-a12a-d06679373232 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:19.595366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8eeed-73af-4532-a8d6-d119e9654fd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:19.596312 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8eeed-73af-4532-a8d6-d119e9654fd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.596490 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8eeed-73af-4532-a8d6-d119e9654fd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.596958 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8eeed-73af-4532-a8d6-d119e9654fd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.599137 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7dc8eeed-73af-4532-a8d6-d119e9654fd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:19.610862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4b395961baa4e6dbcefa46e45145a45'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:19.622190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:19.630721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1320/2631] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:19 2022] DELETE /identity/v3/users/c4b395961baa4e6dbcefa46e45145a45 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:19.634654 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:19.637291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:19.683568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8eeed-73af-4532-a8d6-d119e9654fd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:19.685363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8eeed-73af-4532-a8d6-d119e9654fd7 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:19.696812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1312/2632] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:19.698227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfb9d5-d02d-41fc-9033-3875e2e90d99 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.698616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfb9d5-d02d-41fc-9033-3875e2e90d99 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.698998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfb9d5-d02d-41fc-9033-3875e2e90d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.711212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf9c95-bfa9-4779-8b52-4d1987bb1e25 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:19.716294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c221fc-7b2d-40c6-a5bd-74716232d2ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:19.717045 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c221fc-7b2d-40c6-a5bd-74716232d2ab service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.717340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c221fc-7b2d-40c6-a5bd-74716232d2ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.717617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c221fc-7b2d-40c6-a5bd-74716232d2ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.719740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-73c221fc-7b2d-40c6-a5bd-74716232d2ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:19.724576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-58bfb9d5-d02d-41fc-9033-3875e2e90d99 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:19.726305 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-58bfb9d5-d02d-41fc-9033-3875e2e90d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:19.786817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c221fc-7b2d-40c6-a5bd-74716232d2ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:19.788300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c221fc-7b2d-40c6-a5bd-74716232d2ab service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:19.798688 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1321/2633] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:19.802794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1313/2634] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:19.812080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c221fc-7b2d-40c6-a5bd-74716232d2ab service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:19.813930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-73c221fc-7b2d-40c6-a5bd-74716232d2ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:19.886814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c221fc-7b2d-40c6-a5bd-74716232d2ab service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:19.891203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198bd12-35f5-439b-af71-523be3866e98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:19.891840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198bd12-35f5-439b-af71-523be3866e98 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:19.892109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198bd12-35f5-439b-af71-523be3866e98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:19.892464 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198bd12-35f5-439b-af71-523be3866e98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:19.902173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198bd12-35f5-439b-af71-523be3866e98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:19.903619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198bd12-35f5-439b-af71-523be3866e98 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:19.915292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1322/2635] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:40:20.421169 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfb9d5-d02d-41fc-9033-3875e2e90d99 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:20.422881 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-58bfb9d5-d02d-41fc-9033-3875e2e90d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:20.435712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfb9d5-d02d-41fc-9033-3875e2e90d99 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:20.441085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98455bfb-4404-412b-b1ca-7c88bd1b5e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:20.441727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98455bfb-4404-412b-b1ca-7c88bd1b5e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:20.442007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98455bfb-4404-412b-b1ca-7c88bd1b5e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:20.442432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98455bfb-4404-412b-b1ca-7c88bd1b5e04 admin admin] PATH_INFO: `/v3/projects/e9b773b032444be2b94f3d6ec241f800` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:20.443563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98455bfb-4404-412b-b1ca-7c88bd1b5e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9b773b032444be2b94f3d6ec241f800)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:20.445531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98455bfb-4404-412b-b1ca-7c88bd1b5e04 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:20.448784 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:20.500778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-98455bfb-4404-412b-b1ca-7c88bd1b5e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9b773b032444be2b94f3d6ec241f800', 'request_id': 'req-98455bfb-4404-412b-b1ca-7c88bd1b5e04'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:20.506693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-98455bfb-4404-412b-b1ca-7c88bd1b5e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9b773b032444be2b94f3d6ec241f800', 'request_id': 'req-98455bfb-4404-412b-b1ca-7c88bd1b5e04'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:20.513702 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-98455bfb-4404-412b-b1ca-7c88bd1b5e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9b773b032444be2b94f3d6ec241f800', 'request_id': 'req-98455bfb-4404-412b-b1ca-7c88bd1b5e04'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:20.527922 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-98455bfb-4404-412b-b1ca-7c88bd1b5e04 admin admin] The token cache is being invalidate because project e9b773b032444be2b94f3d6ec241f800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:20.528220 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-98455bfb-4404-412b-b1ca-7c88bd1b5e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:20.528905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-98455bfb-4404-412b-b1ca-7c88bd1b5e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:20.530459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1314/2636] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:20 2022] DELETE /identity/v3/projects/e9b773b032444be2b94f3d6ec241f800 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:20.535703 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffd65f-d2fe-4e68-953e-0286d8c201c1 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:20.536171 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffd65f-d2fe-4e68-953e-0286d8c201c1 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:20.536341 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffd65f-d2fe-4e68-953e-0286d8c201c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:20.557427 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-06ffd65f-d2fe-4e68-953e-0286d8c201c1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:20.558645 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-06ffd65f-d2fe-4e68-953e-0286d8c201c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:20.608163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:20.629440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1323/2637] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:20.631357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f87759-3d1f-492d-b881-92555b12239e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:20.631744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f87759-3d1f-492d-b881-92555b12239e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:20.632059 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f87759-3d1f-492d-b881-92555b12239e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:20.659673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-81f87759-3d1f-492d-b881-92555b12239e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:20.661052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-81f87759-3d1f-492d-b881-92555b12239e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:20.669990 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1315/2638] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:20.674799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffd65f-d2fe-4e68-953e-0286d8c201c1 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:20.684978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffd65f-d2fe-4e68-953e-0286d8c201c1 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:20.690142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb54c1a-fc3e-458c-946e-8841987c9565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:20.690846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb54c1a-fc3e-458c-946e-8841987c9565 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:20.691193 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb54c1a-fc3e-458c-946e-8841987c9565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:20.691472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb54c1a-fc3e-458c-946e-8841987c9565 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:20.692244 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb54c1a-fc3e-458c-946e-8841987c9565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:40:20.692618 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb54c1a-fc3e-458c-946e-8841987c9565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:20.694358 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb54c1a-fc3e-458c-946e-8841987c9565 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:20.707357 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1324/2639] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:40:20.712641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f87759-3d1f-492d-b881-92555b12239e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:20.721974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f87759-3d1f-492d-b881-92555b12239e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:20.726754 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696c274-e895-41c9-a2b4-36b6e363057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:20.727411 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696c274-e895-41c9-a2b4-36b6e363057a admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:20.727799 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696c274-e895-41c9-a2b4-36b6e363057a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:20.728083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696c274-e895-41c9-a2b4-36b6e363057a admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:20.728737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696c274-e895-41c9-a2b4-36b6e363057a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:20.730916 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696c274-e895-41c9-a2b4-36b6e363057a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:20.759287 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1316/2640] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:20 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:20.763868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb54c1a-fc3e-458c-946e-8841987c9565 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:20.774741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb54c1a-fc3e-458c-946e-8841987c9565 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:20.837503 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b22311-4a45-406d-b56d-726f0feebdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:20.838286 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b22311-4a45-406d-b56d-726f0feebdec admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:20.838569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b22311-4a45-406d-b56d-726f0feebdec admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:20.838843 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b22311-4a45-406d-b56d-726f0feebdec admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:20.839477 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b22311-4a45-406d-b56d-726f0feebdec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:20.841426 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b22311-4a45-406d-b56d-726f0feebdec admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:20.863247 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1325/2641] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:20 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:20.868465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696c274-e895-41c9-a2b4-36b6e363057a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:20.879465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696c274-e895-41c9-a2b4-36b6e363057a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:20.885121 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9359d46-8361-46f0-a2e0-130405f4fdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:20.885737 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9359d46-8361-46f0-a2e0-130405f4fdaf admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:20.886020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9359d46-8361-46f0-a2e0-130405f4fdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:20.886298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9359d46-8361-46f0-a2e0-130405f4fdaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:20.886953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9359d46-8361-46f0-a2e0-130405f4fdaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:20.888558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9359d46-8361-46f0-a2e0-130405f4fdaf admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:20.903317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1317/2642] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:40:20.904476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b22311-4a45-406d-b56d-726f0feebdec admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:20.914331 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b22311-4a45-406d-b56d-726f0feebdec admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:20.918970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebdf88-55fb-4985-bd3b-91b426da6642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:20.919575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebdf88-55fb-4985-bd3b-91b426da6642 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:20.919853 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebdf88-55fb-4985-bd3b-91b426da6642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:20.920125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebdf88-55fb-4985-bd3b-91b426da6642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:20.920788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebdf88-55fb-4985-bd3b-91b426da6642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:20.922360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebdf88-55fb-4985-bd3b-91b426da6642 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:20.935624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1326/2643] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:20.939778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9359d46-8361-46f0-a2e0-130405f4fdaf admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:20.951197 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9359d46-8361-46f0-a2e0-130405f4fdaf admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:20.956853 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7477f9c-3d66-4185-a558-ba65ed593670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:20.957560 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7477f9c-3d66-4185-a558-ba65ed593670 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:20.957862 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7477f9c-3d66-4185-a558-ba65ed593670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:20.958113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7477f9c-3d66-4185-a558-ba65ed593670 admin admin] PATH_INFO: `/v3/projects/408cb52625be4bef8d35700ee176c35b/users/e815f7dae93345e5874075b8d0cabeda/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:20.975539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7477f9c-3d66-4185-a558-ba65ed593670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=408cb52625be4bef8d35700ee176c35b, user_id=e815f7dae93345e5874075b8d0cabeda, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:20.978519 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7477f9c-3d66-4185-a558-ba65ed593670 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:20.995678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1318/2644] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:20 2022] PUT /identity/v3/projects/408cb52625be4bef8d35700ee176c35b/users/e815f7dae93345e5874075b8d0cabeda/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:21.006223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489adcb8-71d8-4661-a00c-abbd8be2a5c7 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:21.006591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489adcb8-71d8-4661-a00c-abbd8be2a5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:21.007027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489adcb8-71d8-4661-a00c-abbd8be2a5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:21.040683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-489adcb8-71d8-4661-a00c-abbd8be2a5c7 None None] MFA Rules not processed for user `e815f7dae93345e5874075b8d0cabeda`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:21.042335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-489adcb8-71d8-4661-a00c-abbd8be2a5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:21.175934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1327/2645] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:40:21.177452 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7477f9c-3d66-4185-a558-ba65ed593670 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:21.373394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7477f9c-3d66-4185-a558-ba65ed593670 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:21.375487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb6eb9-ce4c-42b2-b914-39459d0cf67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:21.376412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb6eb9-ce4c-42b2-b914-39459d0cf67b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:21.376904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb6eb9-ce4c-42b2-b914-39459d0cf67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:21.377350 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb6eb9-ce4c-42b2-b914-39459d0cf67b admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:21.378700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb6eb9-ce4c-42b2-b914-39459d0cf67b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:21.381077 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb6eb9-ce4c-42b2-b914-39459d0cf67b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:21.425503 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1319/2646] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:21 2022] POST /identity/v3/projects => generated 377 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:21.429345 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489adcb8-71d8-4661-a00c-abbd8be2a5c7 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:21.446731 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489adcb8-71d8-4661-a00c-abbd8be2a5c7 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:21.501085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:21.519676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159d6f5-aca0-4691-af36-0521ea176a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:21.520715 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159d6f5-aca0-4691-af36-0521ea176a58 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:21.521179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159d6f5-aca0-4691-af36-0521ea176a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:21.521564 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159d6f5-aca0-4691-af36-0521ea176a58 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:21.522257 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159d6f5-aca0-4691-af36-0521ea176a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:21.524892 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159d6f5-aca0-4691-af36-0521ea176a58 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:21.548959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1328/2647] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:21 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:40:21.552835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb6eb9-ce4c-42b2-b914-39459d0cf67b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:21.565728 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb6eb9-ce4c-42b2-b914-39459d0cf67b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:21.571397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ef327-56d6-4cb1-8254-d7ba5ee01b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:21.572089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ef327-56d6-4cb1-8254-d7ba5ee01b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:21.572377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ef327-56d6-4cb1-8254-d7ba5ee01b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:21.573431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ef327-56d6-4cb1-8254-d7ba5ee01b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:21.574124 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ef327-56d6-4cb1-8254-d7ba5ee01b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:21.576039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ef327-56d6-4cb1-8254-d7ba5ee01b4f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:21.591935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1320/2648] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:21.596500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159d6f5-aca0-4691-af36-0521ea176a58 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:21.609357 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159d6f5-aca0-4691-af36-0521ea176a58 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:21.616086 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a8b33-f6d4-4aac-8046-bdeefe43be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:21.616698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a8b33-f6d4-4aac-8046-bdeefe43be3b admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:21.617006 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a8b33-f6d4-4aac-8046-bdeefe43be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:21.617275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a8b33-f6d4-4aac-8046-bdeefe43be3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:21.618468 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a8b33-f6d4-4aac-8046-bdeefe43be3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:21.620243 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a8b33-f6d4-4aac-8046-bdeefe43be3b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:21.631603 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1329/2649] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:21.635717 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ef327-56d6-4cb1-8254-d7ba5ee01b4f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:21.646549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:21.666887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ef327-56d6-4cb1-8254-d7ba5ee01b4f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:21.667641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3480e-491b-4bca-b816-3c737ed9b164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:21.668438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3480e-491b-4bca-b816-3c737ed9b164 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:21.669338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3480e-491b-4bca-b816-3c737ed9b164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:21.670793 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3480e-491b-4bca-b816-3c737ed9b164 admin admin] PATH_INFO: `/v3/projects/005f1c4e3b6241c78e12e5baf7386f41/users/cddfb2e637484e93a8921381aefb398b/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:21.688231 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3480e-491b-4bca-b816-3c737ed9b164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=005f1c4e3b6241c78e12e5baf7386f41, user_id=cddfb2e637484e93a8921381aefb398b, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:21.691369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3480e-491b-4bca-b816-3c737ed9b164 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:21.706306 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1321/2650] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:21 2022] PUT /identity/v3/projects/005f1c4e3b6241c78e12e5baf7386f41/users/cddfb2e637484e93a8921381aefb398b/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:21.718479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f5d3e-bce2-4644-8f1a-1c9d6458f2e5 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:21.718725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f5d3e-bce2-4644-8f1a-1c9d6458f2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:21.718997 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f5d3e-bce2-4644-8f1a-1c9d6458f2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:21.753049 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-f62f5d3e-bce2-4644-8f1a-1c9d6458f2e5 None None] MFA Rules not processed for user `cddfb2e637484e93a8921381aefb398b`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:21.754375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f62f5d3e-bce2-4644-8f1a-1c9d6458f2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:21.843506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1330/2651] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:21.849835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3480e-491b-4bca-b816-3c737ed9b164 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:21.921109 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3480e-491b-4bca-b816-3c737ed9b164 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:21.926033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98efe1-8991-4c0e-9559-13d461a0a2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:21.926741 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98efe1-8991-4c0e-9559-13d461a0a2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:21.927101 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98efe1-8991-4c0e-9559-13d461a0a2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:21.927570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98efe1-8991-4c0e-9559-13d461a0a2e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:21.928283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98efe1-8991-4c0e-9559-13d461a0a2e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:21.930729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98efe1-8991-4c0e-9559-13d461a0a2e0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:21.962424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1322/2652] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:21 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:21.965999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f5d3e-bce2-4644-8f1a-1c9d6458f2e5 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:21.978038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f5d3e-bce2-4644-8f1a-1c9d6458f2e5 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:22.053311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bd44c-5884-44fe-9e10-a54bddc162c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:22.054410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bd44c-5884-44fe-9e10-a54bddc162c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:22.055187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bd44c-5884-44fe-9e10-a54bddc162c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:22.055598 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bd44c-5884-44fe-9e10-a54bddc162c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:22.056524 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51bd44c-5884-44fe-9e10-a54bddc162c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:22.058796 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51bd44c-5884-44fe-9e10-a54bddc162c5 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:22.088027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1331/2653] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:21 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:40:22.089331 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98efe1-8991-4c0e-9559-13d461a0a2e0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:22.101648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98efe1-8991-4c0e-9559-13d461a0a2e0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:22.106797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dc03b-1d5a-4e59-9259-86de8fc15969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:22.107486 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dc03b-1d5a-4e59-9259-86de8fc15969 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:22.107910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dc03b-1d5a-4e59-9259-86de8fc15969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:22.108271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dc03b-1d5a-4e59-9259-86de8fc15969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:22.109149 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dc03b-1d5a-4e59-9259-86de8fc15969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:22.110840 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dc03b-1d5a-4e59-9259-86de8fc15969 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:22.121977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1323/2654] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:22.126460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bd44c-5884-44fe-9e10-a54bddc162c5 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:22.136734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bd44c-5884-44fe-9e10-a54bddc162c5 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:22.141445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa724c-2db4-4b81-aa8b-5640e62f182c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:22.142235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa724c-2db4-4b81-aa8b-5640e62f182c admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:22.142518 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa724c-2db4-4b81-aa8b-5640e62f182c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:22.142793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa724c-2db4-4b81-aa8b-5640e62f182c admin admin] PATH_INFO: `/v3/projects/c7660e7d04354928b9241bf8a09be9d5/users/16d74bf59c29440b903048907b384fbc/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:22.159624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa724c-2db4-4b81-aa8b-5640e62f182c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7660e7d04354928b9241bf8a09be9d5, user_id=16d74bf59c29440b903048907b384fbc, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:22.162402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa724c-2db4-4b81-aa8b-5640e62f182c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:22.176405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1332/2655] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:22 2022] PUT /identity/v3/projects/c7660e7d04354928b9241bf8a09be9d5/users/16d74bf59c29440b903048907b384fbc/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:22.182111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dc03b-1d5a-4e59-9259-86de8fc15969 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:22.192532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dc03b-1d5a-4e59-9259-86de8fc15969 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:22.256695 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1fd0b-fa1b-464f-9bc4-d4f57db23657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:22.257188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1fd0b-fa1b-464f-9bc4-d4f57db23657 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:22.257473 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1fd0b-fa1b-464f-9bc4-d4f57db23657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:22.257851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1fd0b-fa1b-464f-9bc4-d4f57db23657 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:22.258980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1fd0b-fa1b-464f-9bc4-d4f57db23657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:22.260723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1fd0b-fa1b-464f-9bc4-d4f57db23657 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:22.271329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1324/2656] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:22.276596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa724c-2db4-4b81-aa8b-5640e62f182c admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:22.286974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa724c-2db4-4b81-aa8b-5640e62f182c admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:22.291594 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3a6c3-3aa9-474a-8367-30f2b679ecfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:22.292236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3a6c3-3aa9-474a-8367-30f2b679ecfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:22.292512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3a6c3-3aa9-474a-8367-30f2b679ecfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:22.292801 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3a6c3-3aa9-474a-8367-30f2b679ecfd admin admin] PATH_INFO: `/v3/projects/c7660e7d04354928b9241bf8a09be9d5/users/16d74bf59c29440b903048907b384fbc/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:22.294826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3a6c3-3aa9-474a-8367-30f2b679ecfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7660e7d04354928b9241bf8a09be9d5, user_id=16d74bf59c29440b903048907b384fbc, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:22.297638 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3a6c3-3aa9-474a-8367-30f2b679ecfd admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:22.311001 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1333/2657] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:22 2022] PUT /identity/v3/projects/c7660e7d04354928b9241bf8a09be9d5/users/16d74bf59c29440b903048907b384fbc/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:22.321832 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a5bb7-2982-4f54-84d5-4ae6568db626 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:22.322118 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a5bb7-2982-4f54-84d5-4ae6568db626 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:22.322393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a5bb7-2982-4f54-84d5-4ae6568db626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:22.349960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b76a5bb7-2982-4f54-84d5-4ae6568db626 None None] MFA Rules not processed for user `16d74bf59c29440b903048907b384fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:22.351216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b76a5bb7-2982-4f54-84d5-4ae6568db626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:22.390545 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:22.422456 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1325/2658] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:22.433825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3a6c3-3aa9-474a-8367-30f2b679ecfd admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:22.435492 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-02f3a6c3-3aa9-474a-8367-30f2b679ecfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:22.447389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:22.494023 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3a6c3-3aa9-474a-8367-30f2b679ecfd admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:22.497672 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159a598-6344-4e58-919b-92d379affbaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:22.498264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159a598-6344-4e58-919b-92d379affbaf service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:22.498539 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159a598-6344-4e58-919b-92d379affbaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:22.498806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159a598-6344-4e58-919b-92d379affbaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:22.554586 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159a598-6344-4e58-919b-92d379affbaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:22.556787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159a598-6344-4e58-919b-92d379affbaf service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:22.565772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1334/2659] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:40:24.470170 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a5bb7-2982-4f54-84d5-4ae6568db626 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:24.471601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b76a5bb7-2982-4f54-84d5-4ae6568db626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:24.484344 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a5bb7-2982-4f54-84d5-4ae6568db626 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:24.487906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2de88-50aa-4058-b2a5-bea3ece87b7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:24.488511 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2de88-50aa-4058-b2a5-bea3ece87b7f service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:24.488824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2de88-50aa-4058-b2a5-bea3ece87b7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:24.489099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2de88-50aa-4058-b2a5-bea3ece87b7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:24.490795 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-daa2de88-50aa-4058-b2a5-bea3ece87b7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:24.558033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2de88-50aa-4058-b2a5-bea3ece87b7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:24.559479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2de88-50aa-4058-b2a5-bea3ece87b7f service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:24.569730 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1326/2660] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:25.130431 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159a598-6344-4e58-919b-92d379affbaf service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:25.139092 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159a598-6344-4e58-919b-92d379affbaf service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:25.142719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bda711-4a79-4541-945e-2277afd63f77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:25.143311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bda711-4a79-4541-945e-2277afd63f77 service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:25.143585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bda711-4a79-4541-945e-2277afd63f77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:25.143855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bda711-4a79-4541-945e-2277afd63f77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:25.206663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bda711-4a79-4541-945e-2277afd63f77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:25.208139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bda711-4a79-4541-945e-2277afd63f77 service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:25.217653 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1335/2661] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:40:25.592034 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2de88-50aa-4058-b2a5-bea3ece87b7f service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:25.593498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-daa2de88-50aa-4058-b2a5-bea3ece87b7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:25.657942 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2de88-50aa-4058-b2a5-bea3ece87b7f service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:25.662546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:25.663173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:25.663457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:25.663754 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] PATH_INFO: `/v3/users/feaeaf2d6c084c098baaecad6490ade3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:25.665099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feaeaf2d6c084c098baaecad6490ade3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:25.667037 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:25.713892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feaeaf2d6c084c098baaecad6490ade3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:25.714539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feaeaf2d6c084c098baaecad6490ade3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:25.715146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feaeaf2d6c084c098baaecad6490ade3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:25.727338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feaeaf2d6c084c098baaecad6490ade3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:25.731843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:25.735874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feaeaf2d6c084c098baaecad6490ade3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:25.751710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1327/2662] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:25 2022] DELETE /identity/v3/users/feaeaf2d6c084c098baaecad6490ade3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:25.756387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc4c7d-26e4-4625-a960-5882e04ae90d None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:25.756676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc4c7d-26e4-4625-a960-5882e04ae90d None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:25.756992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc4c7d-26e4-4625-a960-5882e04ae90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:25.776144 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-01cc4c7d-26e4-4625-a960-5882e04ae90d None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:25.777376 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-01cc4c7d-26e4-4625-a960-5882e04ae90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:25.835231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1336/2663] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:25.845368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:25.846882 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:25.905602 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7184309-2659-4c2d-aebb-37ff05d920b6 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:25.909631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf2c0a-5ade-4bb5-a6c1-60f9c3d4f2ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:25.910146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf2c0a-5ade-4bb5-a6c1-60f9c3d4f2ef service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:25.910426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf2c0a-5ade-4bb5-a6c1-60f9c3d4f2ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:25.910696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf2c0a-5ade-4bb5-a6c1-60f9c3d4f2ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:25.920136 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf2c0a-5ade-4bb5-a6c1-60f9c3d4f2ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:25.921626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf2c0a-5ade-4bb5-a6c1-60f9c3d4f2ef service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:25.932010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1328/2664] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:26.189621 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc4c7d-26e4-4625-a960-5882e04ae90d None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:26.191068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-01cc4c7d-26e4-4625-a960-5882e04ae90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:26.198240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=100 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:26.204551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc4c7d-26e4-4625-a960-5882e04ae90d None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:26.212198 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:26.215689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:26.215689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:26.215689 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] PATH_INFO: `/v3/projects/3c99be3b3af24f97a9fc25c9ea6f06ef` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:26.216226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c99be3b3af24f97a9fc25c9ea6f06ef)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:26.218225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:26.270911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c99be3b3af24f97a9fc25c9ea6f06ef', 'request_id': 'req-4a3b7e84-2f02-477e-b533-aa24716ad618'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.277379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c99be3b3af24f97a9fc25c9ea6f06ef', 'request_id': 'req-4a3b7e84-2f02-477e-b533-aa24716ad618'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.283065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c99be3b3af24f97a9fc25c9ea6f06ef', 'request_id': 'req-4a3b7e84-2f02-477e-b533-aa24716ad618'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.297860 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] The token cache is being invalidate because project 3c99be3b3af24f97a9fc25c9ea6f06ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:26.298229 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.298814 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.300527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1337/2665] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:26 2022] DELETE /identity/v3/projects/3c99be3b3af24f97a9fc25c9ea6f06ef => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:26.304381 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf2c0a-5ade-4bb5-a6c1-60f9c3d4f2ef service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:26.316631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf2c0a-5ade-4bb5-a6c1-60f9c3d4f2ef service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:26.376210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02069e9d-1af3-41d1-ac7c-bb127d58fe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:26.376876 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02069e9d-1af3-41d1-ac7c-bb127d58fe30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:26.377158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02069e9d-1af3-41d1-ac7c-bb127d58fe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:26.377546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02069e9d-1af3-41d1-ac7c-bb127d58fe30 admin admin] PATH_INFO: `/v3/users/5fca198dd89b46d49fbe64783e95479c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:26.378727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02069e9d-1af3-41d1-ac7c-bb127d58fe30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fca198dd89b46d49fbe64783e95479c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:26.380727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02069e9d-1af3-41d1-ac7c-bb127d58fe30 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:26.427774 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-02069e9d-1af3-41d1-ac7c-bb127d58fe30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fca198dd89b46d49fbe64783e95479c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.428358 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-02069e9d-1af3-41d1-ac7c-bb127d58fe30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fca198dd89b46d49fbe64783e95479c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.428986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-02069e9d-1af3-41d1-ac7c-bb127d58fe30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fca198dd89b46d49fbe64783e95479c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.441249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-02069e9d-1af3-41d1-ac7c-bb127d58fe30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fca198dd89b46d49fbe64783e95479c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.448526 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-02069e9d-1af3-41d1-ac7c-bb127d58fe30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fca198dd89b46d49fbe64783e95479c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.461846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1329/2666] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:26 2022] DELETE /identity/v3/users/5fca198dd89b46d49fbe64783e95479c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:26.733687 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:26.735168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:26.750210 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b7e84-2f02-477e-b533-aa24716ad618 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:26.807169 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba769a53-1eda-4b39-a771-b43380d5f6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:26.807794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba769a53-1eda-4b39-a771-b43380d5f6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:26.808067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba769a53-1eda-4b39-a771-b43380d5f6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:26.808335 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba769a53-1eda-4b39-a771-b43380d5f6b3 admin admin] PATH_INFO: `/v3/projects/a988fa00f32742bfa2083c58b007c21b` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:26.809453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba769a53-1eda-4b39-a771-b43380d5f6b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a988fa00f32742bfa2083c58b007c21b)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:26.811340 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba769a53-1eda-4b39-a771-b43380d5f6b3 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:26.857277 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ba769a53-1eda-4b39-a771-b43380d5f6b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a988fa00f32742bfa2083c58b007c21b', 'request_id': 'req-ba769a53-1eda-4b39-a771-b43380d5f6b3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.863233 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ba769a53-1eda-4b39-a771-b43380d5f6b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a988fa00f32742bfa2083c58b007c21b', 'request_id': 'req-ba769a53-1eda-4b39-a771-b43380d5f6b3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.869264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ba769a53-1eda-4b39-a771-b43380d5f6b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a988fa00f32742bfa2083c58b007c21b', 'request_id': 'req-ba769a53-1eda-4b39-a771-b43380d5f6b3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.881283 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ba769a53-1eda-4b39-a771-b43380d5f6b3 admin admin] The token cache is being invalidate because project a988fa00f32742bfa2083c58b007c21b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:26.881584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ba769a53-1eda-4b39-a771-b43380d5f6b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.882179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ba769a53-1eda-4b39-a771-b43380d5f6b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:26.883910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1338/2667] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:26 2022] DELETE /identity/v3/projects/a988fa00f32742bfa2083c58b007c21b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:26.888689 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34f8cc-8c50-43ee-89e7-3f77f9dba103 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:26.889188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34f8cc-8c50-43ee-89e7-3f77f9dba103 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:26.889294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34f8cc-8c50-43ee-89e7-3f77f9dba103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:26.912186 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3b34f8cc-8c50-43ee-89e7-3f77f9dba103 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:26.913462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3b34f8cc-8c50-43ee-89e7-3f77f9dba103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:26.918641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:26.986297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1330/2668] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:26.994854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a8c05-73f4-40d2-b0e6-b5080f94256b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:26.995141 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a8c05-73f4-40d2-b0e6-b5080f94256b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:26.995413 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a8c05-73f4-40d2-b0e6-b5080f94256b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.017079 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-c53a8c05-73f4-40d2-b0e6-b5080f94256b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:27.018326 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c53a8c05-73f4-40d2-b0e6-b5080f94256b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:27.026775 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1339/2669] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:27.031255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34f8cc-8c50-43ee-89e7-3f77f9dba103 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:27.043467 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34f8cc-8c50-43ee-89e7-3f77f9dba103 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:27.048173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4de760-fe9a-47ef-8f9d-614aedc09361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:27.048838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4de760-fe9a-47ef-8f9d-614aedc09361 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:27.049120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4de760-fe9a-47ef-8f9d-614aedc09361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:27.049385 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4de760-fe9a-47ef-8f9d-614aedc09361 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.050026 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4de760-fe9a-47ef-8f9d-614aedc09361 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:40:27.050321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4de760-fe9a-47ef-8f9d-614aedc09361 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:27.051898 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4de760-fe9a-47ef-8f9d-614aedc09361 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:27.064784 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1331/2670] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:40:27.069233 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a8c05-73f4-40d2-b0e6-b5080f94256b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:27.078851 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a8c05-73f4-40d2-b0e6-b5080f94256b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:27.089096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee8eed-d7dc-45dd-b14f-ec38cdf03a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:27.089725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee8eed-d7dc-45dd-b14f-ec38cdf03a5c admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:27.090006 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee8eed-d7dc-45dd-b14f-ec38cdf03a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:27.090306 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee8eed-d7dc-45dd-b14f-ec38cdf03a5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.090978 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee8eed-d7dc-45dd-b14f-ec38cdf03a5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:27.092946 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee8eed-d7dc-45dd-b14f-ec38cdf03a5c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:27.098857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:27.124171 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1340/2671] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:27 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:27.128430 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4de760-fe9a-47ef-8f9d-614aedc09361 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:27.139610 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4de760-fe9a-47ef-8f9d-614aedc09361 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:27.175566 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee8eed-d7dc-45dd-b14f-ec38cdf03a5c admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:27.177281 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-8dee8eed-d7dc-45dd-b14f-ec38cdf03a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:27.199420 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03237-ba82-42c4-949d-db69bc43c6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:27.200069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03237-ba82-42c4-949d-db69bc43c6de admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:27.200368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03237-ba82-42c4-949d-db69bc43c6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:27.200662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03237-ba82-42c4-949d-db69bc43c6de admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.201337 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a03237-ba82-42c4-949d-db69bc43c6de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:27.203369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a03237-ba82-42c4-949d-db69bc43c6de admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:27.225375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1332/2672] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:27 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:27.230059 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03237-ba82-42c4-949d-db69bc43c6de admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:27.239671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03237-ba82-42c4-949d-db69bc43c6de admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:27.240199 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee8eed-d7dc-45dd-b14f-ec38cdf03a5c admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:27.244690 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c6b07-2d9c-4272-9c95-0f6ae9ea5cef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:27.244928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928a708-a6a0-4d72-8822-25ccefa90cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:27.245323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c6b07-2d9c-4272-9c95-0f6ae9ea5cef service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:27.245523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928a708-a6a0-4d72-8822-25ccefa90cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:27.245643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c6b07-2d9c-4272-9c95-0f6ae9ea5cef service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:27.245806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928a708-a6a0-4d72-8822-25ccefa90cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:27.245916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c6b07-2d9c-4272-9c95-0f6ae9ea5cef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.246081 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928a708-a6a0-4d72-8822-25ccefa90cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.246715 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928a708-a6a0-4d72-8822-25ccefa90cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:27.247885 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-217c6b07-2d9c-4272-9c95-0f6ae9ea5cef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:27.248309 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928a708-a6a0-4d72-8822-25ccefa90cb6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:27.259962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1333/2673] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:40:27.263767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928a708-a6a0-4d72-8822-25ccefa90cb6 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:27.273396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928a708-a6a0-4d72-8822-25ccefa90cb6 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:27.278546 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71447556-5352-4f52-a8f6-af3da5f89753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:27.279378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71447556-5352-4f52-a8f6-af3da5f89753 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:27.279662 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71447556-5352-4f52-a8f6-af3da5f89753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:27.280023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71447556-5352-4f52-a8f6-af3da5f89753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.280586 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71447556-5352-4f52-a8f6-af3da5f89753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:27.282202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71447556-5352-4f52-a8f6-af3da5f89753 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:27.292342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1334/2674] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:40:27.296356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71447556-5352-4f52-a8f6-af3da5f89753 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:27.306699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71447556-5352-4f52-a8f6-af3da5f89753 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:27.311558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00b68d-5185-4f92-8d07-ef5d30767d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:27.313028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00b68d-5185-4f92-8d07-ef5d30767d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:27.313308 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00b68d-5185-4f92-8d07-ef5d30767d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:27.313578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00b68d-5185-4f92-8d07-ef5d30767d76 admin admin] PATH_INFO: `/v3/projects/b155a9e572714e0ba175464a569e6bb5/users/f4e972da326d48908f9474fa3d2b1239/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.314680 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c6b07-2d9c-4272-9c95-0f6ae9ea5cef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:27.316165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c6b07-2d9c-4272-9c95-0f6ae9ea5cef service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:27.326976 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1341/2675] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:40:27.329178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00b68d-5185-4f92-8d07-ef5d30767d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b155a9e572714e0ba175464a569e6bb5, user_id=f4e972da326d48908f9474fa3d2b1239, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:27.331826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00b68d-5185-4f92-8d07-ef5d30767d76 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:27.345566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1335/2676] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:27 2022] PUT /identity/v3/projects/b155a9e572714e0ba175464a569e6bb5/users/f4e972da326d48908f9474fa3d2b1239/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:40:27.356309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec4bbb-3c6e-445f-be17-c99433f58242 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:27.356808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec4bbb-3c6e-445f-be17-c99433f58242 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:27.357275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec4bbb-3c6e-445f-be17-c99433f58242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.393748 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d0ec4bbb-3c6e-445f-be17-c99433f58242 None None] MFA Rules not processed for user `f4e972da326d48908f9474fa3d2b1239`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:27.395375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d0ec4bbb-3c6e-445f-be17-c99433f58242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:27.470322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1342/2677] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:27.481630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00b68d-5185-4f92-8d07-ef5d30767d76 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:27.526528 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:27.541715 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00b68d-5185-4f92-8d07-ef5d30767d76 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:27.545352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5707f2-ae5d-46ee-ab4d-7eaad4dbf9ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:27.545961 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5707f2-ae5d-46ee-ab4d-7eaad4dbf9ac service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:27.546238 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5707f2-ae5d-46ee-ab4d-7eaad4dbf9ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:27.546509 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5707f2-ae5d-46ee-ab4d-7eaad4dbf9ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.556229 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5707f2-ae5d-46ee-ab4d-7eaad4dbf9ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:27.557576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5707f2-ae5d-46ee-ab4d-7eaad4dbf9ac service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:27.566910 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1336/2678] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:27.586046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec4bbb-3c6e-445f-be17-c99433f58242 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:27.617060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:27.651635 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec4bbb-3c6e-445f-be17-c99433f58242 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:27.656294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:27.656955 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:27.657238 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:27.657512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] PATH_INFO: `/v3/users/f4e972da326d48908f9474fa3d2b1239` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.658667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4e972da326d48908f9474fa3d2b1239)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:27.660575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:27.707627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4e972da326d48908f9474fa3d2b1239'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:27.708287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4e972da326d48908f9474fa3d2b1239'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:27.708960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4e972da326d48908f9474fa3d2b1239'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:27.715988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4e972da326d48908f9474fa3d2b1239'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:27.728220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4e972da326d48908f9474fa3d2b1239'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:27.740297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1343/2679] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:27 2022] DELETE /identity/v3/users/f4e972da326d48908f9474fa3d2b1239 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:27.746758 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77e05c-82d4-473a-a1b9-52d0126970a6 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:27.747071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77e05c-82d4-473a-a1b9-52d0126970a6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:27.747346 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77e05c-82d4-473a-a1b9-52d0126970a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.765892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6b77e05c-82d4-473a-a1b9-52d0126970a6 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:27.767080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6b77e05c-82d4-473a-a1b9-52d0126970a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:27.825368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1337/2680] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:27.834999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:27.837085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:27.894482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9815f-72ba-4ff3-887c-29b111cf4430 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:27.898125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c8e2a-4945-46a4-8ef5-9427a0709b10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:27.899440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c8e2a-4945-46a4-8ef5-9427a0709b10 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:27.899720 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c8e2a-4945-46a4-8ef5-9427a0709b10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:27.899992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c8e2a-4945-46a4-8ef5-9427a0709b10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:27.909373 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c8e2a-4945-46a4-8ef5-9427a0709b10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:27.910802 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c8e2a-4945-46a4-8ef5-9427a0709b10 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:27.921135 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1344/2681] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:28.195095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77e05c-82d4-473a-a1b9-52d0126970a6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:28.197241 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6b77e05c-82d4-473a-a1b9-52d0126970a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:28.217613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77e05c-82d4-473a-a1b9-52d0126970a6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:28.222512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74502ce9-22b7-416a-9237-f159da8bb7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:28.223156 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74502ce9-22b7-416a-9237-f159da8bb7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:28.223435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74502ce9-22b7-416a-9237-f159da8bb7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:28.223820 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74502ce9-22b7-416a-9237-f159da8bb7c8 admin admin] PATH_INFO: `/v3/projects/b155a9e572714e0ba175464a569e6bb5` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:28.226714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74502ce9-22b7-416a-9237-f159da8bb7c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b155a9e572714e0ba175464a569e6bb5)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:28.227850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74502ce9-22b7-416a-9237-f159da8bb7c8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:28.275671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-74502ce9-22b7-416a-9237-f159da8bb7c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b155a9e572714e0ba175464a569e6bb5', 'request_id': 'req-74502ce9-22b7-416a-9237-f159da8bb7c8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:28.281656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-74502ce9-22b7-416a-9237-f159da8bb7c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b155a9e572714e0ba175464a569e6bb5', 'request_id': 'req-74502ce9-22b7-416a-9237-f159da8bb7c8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:28.287764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-74502ce9-22b7-416a-9237-f159da8bb7c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b155a9e572714e0ba175464a569e6bb5', 'request_id': 'req-74502ce9-22b7-416a-9237-f159da8bb7c8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:28.299957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-74502ce9-22b7-416a-9237-f159da8bb7c8 admin admin] The token cache is being invalidate because project b155a9e572714e0ba175464a569e6bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:28.300246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-74502ce9-22b7-416a-9237-f159da8bb7c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:28.300930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-74502ce9-22b7-416a-9237-f159da8bb7c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:28.302868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1338/2682] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:28 2022] DELETE /identity/v3/projects/b155a9e572714e0ba175464a569e6bb5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:28.307820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8240b63-29b3-4aa6-887e-3e8abae9b3b6 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:28.308239 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8240b63-29b3-4aa6-887e-3e8abae9b3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:28.308399 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8240b63-29b3-4aa6-887e-3e8abae9b3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:28.330290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-a8240b63-29b3-4aa6-887e-3e8abae9b3b6 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:28.332384 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a8240b63-29b3-4aa6-887e-3e8abae9b3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:28.385914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:28.402659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1345/2683] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:28.411678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12cca-d92c-48c9-b605-1c3ba8705909 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:28.411975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12cca-d92c-48c9-b605-1c3ba8705909 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:28.412256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12cca-d92c-48c9-b605-1c3ba8705909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:28.431718 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-33f12cca-d92c-48c9-b605-1c3ba8705909 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:28.432967 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-33f12cca-d92c-48c9-b605-1c3ba8705909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:28.441089 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1339/2684] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:28.446028 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8240b63-29b3-4aa6-887e-3e8abae9b3b6 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:28.459906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8240b63-29b3-4aa6-887e-3e8abae9b3b6 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:28.467237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d072d82-75bf-4fbb-a97d-8df4cab4203a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:28.467835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d072d82-75bf-4fbb-a97d-8df4cab4203a admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:28.468112 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d072d82-75bf-4fbb-a97d-8df4cab4203a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:28.468398 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d072d82-75bf-4fbb-a97d-8df4cab4203a admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:28.469054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d072d82-75bf-4fbb-a97d-8df4cab4203a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:40:28.469347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d072d82-75bf-4fbb-a97d-8df4cab4203a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:28.470933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d072d82-75bf-4fbb-a97d-8df4cab4203a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:28.483434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1346/2685] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:40:28.487571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12cca-d92c-48c9-b605-1c3ba8705909 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:28.497270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12cca-d92c-48c9-b605-1c3ba8705909 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:28.501896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b0180-11ee-4197-a585-87de5beaeeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:28.502578 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b0180-11ee-4197-a585-87de5beaeeec admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:28.502854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b0180-11ee-4197-a585-87de5beaeeec admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:28.503120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b0180-11ee-4197-a585-87de5beaeeec admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:28.503745 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b0180-11ee-4197-a585-87de5beaeeec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:28.505678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b0180-11ee-4197-a585-87de5beaeeec admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:28.535375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1340/2686] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:28 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:40:28.539461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d072d82-75bf-4fbb-a97d-8df4cab4203a admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:28.549739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d072d82-75bf-4fbb-a97d-8df4cab4203a admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:28.615214 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d483de-504f-46ed-bdd6-ec8b2136806c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:28.615951 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d483de-504f-46ed-bdd6-ec8b2136806c admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:28.616346 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d483de-504f-46ed-bdd6-ec8b2136806c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:28.616605 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d483de-504f-46ed-bdd6-ec8b2136806c admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:28.617488 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d483de-504f-46ed-bdd6-ec8b2136806c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:28.619651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d483de-504f-46ed-bdd6-ec8b2136806c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:28.644324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1347/2687] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:28 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:40:28.648973 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b0180-11ee-4197-a585-87de5beaeeec admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:28.670554 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b0180-11ee-4197-a585-87de5beaeeec admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:28.675868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e58f0-1299-43db-9b7b-6faeb0c7669b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:28.676618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e58f0-1299-43db-9b7b-6faeb0c7669b admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:28.677027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e58f0-1299-43db-9b7b-6faeb0c7669b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:28.677375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e58f0-1299-43db-9b7b-6faeb0c7669b admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:28.678286 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e58f0-1299-43db-9b7b-6faeb0c7669b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:28.680212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e58f0-1299-43db-9b7b-6faeb0c7669b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:28.692801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1341/2688] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:28.696632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d483de-504f-46ed-bdd6-ec8b2136806c admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:28.711524 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d483de-504f-46ed-bdd6-ec8b2136806c admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:28.721460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6db22-c19e-47e0-8f72-343ff1dcc29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:28.722791 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6db22-c19e-47e0-8f72-343ff1dcc29c admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:28.723069 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6db22-c19e-47e0-8f72-343ff1dcc29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:28.723344 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6db22-c19e-47e0-8f72-343ff1dcc29c admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:28.724076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a6db22-c19e-47e0-8f72-343ff1dcc29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:28.726051 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a6db22-c19e-47e0-8f72-343ff1dcc29c admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:28.740792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1348/2689] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:28.745057 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e58f0-1299-43db-9b7b-6faeb0c7669b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:28.758352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e58f0-1299-43db-9b7b-6faeb0c7669b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:28.763801 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc2261-0e0e-41d5-ad2d-d15140b2ca8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:28.764501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc2261-0e0e-41d5-ad2d-d15140b2ca8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:28.764815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc2261-0e0e-41d5-ad2d-d15140b2ca8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:28.765099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc2261-0e0e-41d5-ad2d-d15140b2ca8b admin admin] PATH_INFO: `/v3/projects/17acb86dbdc24b92b42c368e3cc1af28/users/0f5756987a42401c816a24e3173f40ae/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:28.785537 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc2261-0e0e-41d5-ad2d-d15140b2ca8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=17acb86dbdc24b92b42c368e3cc1af28, user_id=0f5756987a42401c816a24e3173f40ae, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:28.788569 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc2261-0e0e-41d5-ad2d-d15140b2ca8b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:28.806932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1342/2690] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:28 2022] PUT /identity/v3/projects/17acb86dbdc24b92b42c368e3cc1af28/users/0f5756987a42401c816a24e3173f40ae/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:28.817845 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192848e5-305f-4b6b-a98c-0efda8b495d5 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:28.818235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192848e5-305f-4b6b-a98c-0efda8b495d5 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:28.818571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192848e5-305f-4b6b-a98c-0efda8b495d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:28.847367 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-192848e5-305f-4b6b-a98c-0efda8b495d5 None None] MFA Rules not processed for user `0f5756987a42401c816a24e3173f40ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:28.848592 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-192848e5-305f-4b6b-a98c-0efda8b495d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:28.913033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1349/2691] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:28.924693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc2261-0e0e-41d5-ad2d-d15140b2ca8b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:28.926475 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2ffc2261-0e0e-41d5-ad2d-d15140b2ca8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:28.932365 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:28.986396 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc2261-0e0e-41d5-ad2d-d15140b2ca8b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:28.990094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20cd32-0928-4b24-9b56-9eeab6e66890 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '748ad823b944441d907994cda6d8ac7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:28.990703 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20cd32-0928-4b24-9b56-9eeab6e66890 service glance] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:28.990986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20cd32-0928-4b24-9b56-9eeab6e66890 service glance] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:28.991264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20cd32-0928-4b24-9b56-9eeab6e66890 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:29.000227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20cd32-0928-4b24-9b56-9eeab6e66890 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:29.001860 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20cd32-0928-4b24-9b56-9eeab6e66890 service glance] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:29.011225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1343/2692] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:31.290505 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192848e5-305f-4b6b-a98c-0efda8b495d5 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:31.291988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-192848e5-305f-4b6b-a98c-0efda8b495d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:31.338551 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:31.360347 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192848e5-305f-4b6b-a98c-0efda8b495d5 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:31.364826 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:31.365461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:31.365736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:31.366005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] PATH_INFO: `/v3/users/e815f7dae93345e5874075b8d0cabeda` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:31.367153 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e815f7dae93345e5874075b8d0cabeda)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:31.369116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:31.416914 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e815f7dae93345e5874075b8d0cabeda'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:31.417700 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e815f7dae93345e5874075b8d0cabeda'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:31.418419 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e815f7dae93345e5874075b8d0cabeda'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:31.426127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e815f7dae93345e5874075b8d0cabeda'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:31.438658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e815f7dae93345e5874075b8d0cabeda'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:31.451749 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1350/2693] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:31 2022] DELETE /identity/v3/users/e815f7dae93345e5874075b8d0cabeda => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:31.456233 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf7a7c-192a-4c8c-b515-7a2d5b54d6a1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:31.456712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf7a7c-192a-4c8c-b515-7a2d5b54d6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:31.456865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf7a7c-192a-4c8c-b515-7a2d5b54d6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:31.476191 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ccbf7a7c-192a-4c8c-b515-7a2d5b54d6a1 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:31.477550 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ccbf7a7c-192a-4c8c-b515-7a2d5b54d6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:31.545395 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1344/2694] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:31.556078 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:31.557582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:31.614984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd54f1-22e3-4e1f-a451-93f0a42d704e admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:31.618772 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76d2d6-0d19-4c27-bb68-21bcf3e792ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:31.619377 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76d2d6-0d19-4c27-bb68-21bcf3e792ba service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:31.619660 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76d2d6-0d19-4c27-bb68-21bcf3e792ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:31.619933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76d2d6-0d19-4c27-bb68-21bcf3e792ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:31.628831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76d2d6-0d19-4c27-bb68-21bcf3e792ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:31.630260 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76d2d6-0d19-4c27-bb68-21bcf3e792ba service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:31.639870 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1351/2695] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:31.906743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf7a7c-192a-4c8c-b515-7a2d5b54d6a1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:31.908498 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ccbf7a7c-192a-4c8c-b515-7a2d5b54d6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:31.921685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf7a7c-192a-4c8c-b515-7a2d5b54d6a1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:31.926493 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:31.927125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:31.927405 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:31.927678 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] PATH_INFO: `/v3/projects/408cb52625be4bef8d35700ee176c35b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:31.928817 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] RBAC: Authorizing `identity:delete_project(project_id=408cb52625be4bef8d35700ee176c35b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:31.930843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:31.977982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '408cb52625be4bef8d35700ee176c35b', 'request_id': 'req-153e5a1e-0103-482a-9b78-7dcdb541706e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:31.983806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '408cb52625be4bef8d35700ee176c35b', 'request_id': 'req-153e5a1e-0103-482a-9b78-7dcdb541706e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:31.989941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '408cb52625be4bef8d35700ee176c35b', 'request_id': 'req-153e5a1e-0103-482a-9b78-7dcdb541706e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.002047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] The token cache is being invalidate because project 408cb52625be4bef8d35700ee176c35b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:32.002338 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.002963 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.004485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1345/2696] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:31 2022] DELETE /identity/v3/projects/408cb52625be4bef8d35700ee176c35b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:32.009167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76d2d6-0d19-4c27-bb68-21bcf3e792ba service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:32.018693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76d2d6-0d19-4c27-bb68-21bcf3e792ba service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:32.074684 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:32.075295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:32.075571 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:32.075840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] PATH_INFO: `/v3/users/cddfb2e637484e93a8921381aefb398b` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:32.076967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cddfb2e637484e93a8921381aefb398b)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:32.078851 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:32.083975 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:32.123864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cddfb2e637484e93a8921381aefb398b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.124563 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cddfb2e637484e93a8921381aefb398b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.125215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cddfb2e637484e93a8921381aefb398b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.132063 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cddfb2e637484e93a8921381aefb398b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.144083 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cddfb2e637484e93a8921381aefb398b'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.157898 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1352/2697] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:32 2022] DELETE /identity/v3/users/cddfb2e637484e93a8921381aefb398b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:32.513739 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:32.515421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:32.528738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e5a1e-0103-482a-9b78-7dcdb541706e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:32.556378 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:32.590181 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:32.590825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:32.591108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:32.591382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] PATH_INFO: `/v3/projects/005f1c4e3b6241c78e12e5baf7386f41` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:32.592536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] RBAC: Authorizing `identity:delete_project(project_id=005f1c4e3b6241c78e12e5baf7386f41)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:32.594518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:32.641742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '005f1c4e3b6241c78e12e5baf7386f41', 'request_id': 'req-3f34d72e-9d1c-41a4-8aec-daae0da9139b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.647315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '005f1c4e3b6241c78e12e5baf7386f41', 'request_id': 'req-3f34d72e-9d1c-41a4-8aec-daae0da9139b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.653736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '005f1c4e3b6241c78e12e5baf7386f41', 'request_id': 'req-3f34d72e-9d1c-41a4-8aec-daae0da9139b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.666360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] The token cache is being invalidate because project 005f1c4e3b6241c78e12e5baf7386f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:32.666671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.667272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.668746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1346/2698] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:32 2022] DELETE /identity/v3/projects/005f1c4e3b6241c78e12e5baf7386f41 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:32.672908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:32.674387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:32.688309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27ca81-e958-4338-96e6-a289ad159f35 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:32.744874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e4953-3a45-4e36-834a-aa32c5b40503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:32.745500 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e4953-3a45-4e36-834a-aa32c5b40503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:32.745784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e4953-3a45-4e36-834a-aa32c5b40503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:32.746060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e4953-3a45-4e36-834a-aa32c5b40503 admin admin] PATH_INFO: `/v3/users/16d74bf59c29440b903048907b384fbc` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:32.747165 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e4953-3a45-4e36-834a-aa32c5b40503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d74bf59c29440b903048907b384fbc)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:32.749119 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e4953-3a45-4e36-834a-aa32c5b40503 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:32.794709 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-638e4953-3a45-4e36-834a-aa32c5b40503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d74bf59c29440b903048907b384fbc'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.795503 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-638e4953-3a45-4e36-834a-aa32c5b40503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d74bf59c29440b903048907b384fbc'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.796304 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-638e4953-3a45-4e36-834a-aa32c5b40503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d74bf59c29440b903048907b384fbc'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.803969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-638e4953-3a45-4e36-834a-aa32c5b40503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d74bf59c29440b903048907b384fbc'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.816602 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-638e4953-3a45-4e36-834a-aa32c5b40503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d74bf59c29440b903048907b384fbc'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:32.828958 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1353/2699] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:32 2022] DELETE /identity/v3/users/16d74bf59c29440b903048907b384fbc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:33.099250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:33.100711 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:33.114663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34d72e-9d1c-41a4-8aec-daae0da9139b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:33.173011 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f423b8e-b741-4bf7-8912-2e763f07f311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:33.173647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f423b8e-b741-4bf7-8912-2e763f07f311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:33.173935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f423b8e-b741-4bf7-8912-2e763f07f311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:33.174209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f423b8e-b741-4bf7-8912-2e763f07f311 admin admin] PATH_INFO: `/v3/projects/c7660e7d04354928b9241bf8a09be9d5` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:33.175317 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f423b8e-b741-4bf7-8912-2e763f07f311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7660e7d04354928b9241bf8a09be9d5)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:33.177250 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f423b8e-b741-4bf7-8912-2e763f07f311 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:33.223918 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2f423b8e-b741-4bf7-8912-2e763f07f311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7660e7d04354928b9241bf8a09be9d5', 'request_id': 'req-2f423b8e-b741-4bf7-8912-2e763f07f311'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:33.230132 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2f423b8e-b741-4bf7-8912-2e763f07f311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7660e7d04354928b9241bf8a09be9d5', 'request_id': 'req-2f423b8e-b741-4bf7-8912-2e763f07f311'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:33.232011 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:33.237485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2f423b8e-b741-4bf7-8912-2e763f07f311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7660e7d04354928b9241bf8a09be9d5', 'request_id': 'req-2f423b8e-b741-4bf7-8912-2e763f07f311'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:33.250382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2f423b8e-b741-4bf7-8912-2e763f07f311 admin admin] The token cache is being invalidate because project c7660e7d04354928b9241bf8a09be9d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:33.250672 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2f423b8e-b741-4bf7-8912-2e763f07f311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:33.251410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2f423b8e-b741-4bf7-8912-2e763f07f311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:33.252904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1347/2700] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:33 2022] DELETE /identity/v3/projects/c7660e7d04354928b9241bf8a09be9d5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:33.258884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c7d91-4030-4fa2-8612-4ab9347e0eeb None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:33.259167 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c7d91-4030-4fa2-8612-4ab9347e0eeb None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:33.259440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c7d91-4030-4fa2-8612-4ab9347e0eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:33.267799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:33.279974 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-221c7d91-4030-4fa2-8612-4ab9347e0eeb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:33.281367 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-221c7d91-4030-4fa2-8612-4ab9347e0eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:33.344906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1354/2701] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:40:33.353570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f81c2-36a0-49f6-add5-999a829abd5e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:33.353945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f81c2-36a0-49f6-add5-999a829abd5e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:33.354225 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f81c2-36a0-49f6-add5-999a829abd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:33.374682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-174f81c2-36a0-49f6-add5-999a829abd5e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:33.375925 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-174f81c2-36a0-49f6-add5-999a829abd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:33.384032 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1348/2702] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:33.388405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c7d91-4030-4fa2-8612-4ab9347e0eeb None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:33.398725 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c7d91-4030-4fa2-8612-4ab9347e0eeb None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:33.404886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ed56f-2021-4646-a14e-cde5784329e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:33.405593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ed56f-2021-4646-a14e-cde5784329e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:33.405984 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ed56f-2021-4646-a14e-cde5784329e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:33.406264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ed56f-2021-4646-a14e-cde5784329e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:33.409653 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ed56f-2021-4646-a14e-cde5784329e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:40:33.410440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ed56f-2021-4646-a14e-cde5784329e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:33.416045 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ed56f-2021-4646-a14e-cde5784329e8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:33.429125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1355/2703] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:40:33.433595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f81c2-36a0-49f6-add5-999a829abd5e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:33.444272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f81c2-36a0-49f6-add5-999a829abd5e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:33.450784 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d68843-6b8f-4529-a402-da3fcb813aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:33.451723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d68843-6b8f-4529-a402-da3fcb813aad admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:33.452165 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d68843-6b8f-4529-a402-da3fcb813aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:33.452600 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d68843-6b8f-4529-a402-da3fcb813aad admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:33.453505 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d68843-6b8f-4529-a402-da3fcb813aad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:33.455620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d68843-6b8f-4529-a402-da3fcb813aad admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:33.488001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1349/2704] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:33 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:40:33.491916 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ed56f-2021-4646-a14e-cde5784329e8 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:33.504971 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ed56f-2021-4646-a14e-cde5784329e8 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:33.597149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168486fd-0315-445c-88f0-a355f3c429d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:33.598513 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168486fd-0315-445c-88f0-a355f3c429d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:33.598809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168486fd-0315-445c-88f0-a355f3c429d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:33.599084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168486fd-0315-445c-88f0-a355f3c429d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:33.599758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168486fd-0315-445c-88f0-a355f3c429d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:33.602436 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168486fd-0315-445c-88f0-a355f3c429d4 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:33.635061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1356/2705] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:33 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:33.635960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d68843-6b8f-4529-a402-da3fcb813aad admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:33.650190 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d68843-6b8f-4529-a402-da3fcb813aad admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:33.659548 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac8f02-630f-49eb-903c-34106a25beec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:33.660325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac8f02-630f-49eb-903c-34106a25beec admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:33.660685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac8f02-630f-49eb-903c-34106a25beec admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:33.661076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac8f02-630f-49eb-903c-34106a25beec admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:33.661856 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac8f02-630f-49eb-903c-34106a25beec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:33.664113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac8f02-630f-49eb-903c-34106a25beec admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:33.680280 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168486fd-0315-445c-88f0-a355f3c429d4 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:33.688945 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1350/2706] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:33.699215 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168486fd-0315-445c-88f0-a355f3c429d4 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:33.705533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd0863-9d21-4952-b4ac-e9971eb8094f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:33.706146 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd0863-9d21-4952-b4ac-e9971eb8094f admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:33.706479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd0863-9d21-4952-b4ac-e9971eb8094f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:33.706758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd0863-9d21-4952-b4ac-e9971eb8094f admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:33.707443 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd0863-9d21-4952-b4ac-e9971eb8094f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:33.709305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd0863-9d21-4952-b4ac-e9971eb8094f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:33.722398 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1357/2707] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:33.729617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac8f02-630f-49eb-903c-34106a25beec admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:33.746258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac8f02-630f-49eb-903c-34106a25beec admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:33.755431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67a26e-e574-4cf1-bde7-fbca3af66233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:33.756158 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67a26e-e574-4cf1-bde7-fbca3af66233 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:33.756474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67a26e-e574-4cf1-bde7-fbca3af66233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:33.756849 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67a26e-e574-4cf1-bde7-fbca3af66233 admin admin] PATH_INFO: `/v3/projects/a120d2cba9884f148185606778440fff/users/5f0dd23f0f0b4891a9933cbe8a4861f3/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:33.774996 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67a26e-e574-4cf1-bde7-fbca3af66233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a120d2cba9884f148185606778440fff, user_id=5f0dd23f0f0b4891a9933cbe8a4861f3, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:33.777822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67a26e-e574-4cf1-bde7-fbca3af66233 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:33.792291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1351/2708] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:33 2022] PUT /identity/v3/projects/a120d2cba9884f148185606778440fff/users/5f0dd23f0f0b4891a9933cbe8a4861f3/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:33.803578 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cb263-d644-43ab-b0d1-e574a6283f83 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:33.803865 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cb263-d644-43ab-b0d1-e574a6283f83 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:33.804666 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cb263-d644-43ab-b0d1-e574a6283f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:33.833229 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-a91cb263-d644-43ab-b0d1-e574a6283f83 None None] MFA Rules not processed for user `5f0dd23f0f0b4891a9933cbe8a4861f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:33.834484 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-a91cb263-d644-43ab-b0d1-e574a6283f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:33.900311 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:33.904324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1358/2709] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:33.915806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67a26e-e574-4cf1-bde7-fbca3af66233 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:33.917552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0f67a26e-e574-4cf1-bde7-fbca3af66233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:33.978573 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67a26e-e574-4cf1-bde7-fbca3af66233 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:33.982616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f779cbb-b8af-48b3-b735-19060e59927c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:33.983224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f779cbb-b8af-48b3-b735-19060e59927c service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:33.983501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f779cbb-b8af-48b3-b735-19060e59927c service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:33.983773 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f779cbb-b8af-48b3-b735-19060e59927c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:33.993331 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f779cbb-b8af-48b3-b735-19060e59927c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:33.994772 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f779cbb-b8af-48b3-b735-19060e59927c service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:34.003618 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1352/2710] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:40:35.392175 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f855d-f743-41eb-a62e-32f45fa5d296 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:35.392639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f855d-f743-41eb-a62e-32f45fa5d296 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:35.392739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f855d-f743-41eb-a62e-32f45fa5d296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:35.412582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d97f855d-f743-41eb-a62e-32f45fa5d296 None None] MFA Rules not processed for user `5f0dd23f0f0b4891a9933cbe8a4861f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:35.414669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d97f855d-f743-41eb-a62e-32f45fa5d296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:35.422343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1359/2711] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:43.999624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=56,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:45.406593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:48.626936 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f779cbb-b8af-48b3-b735-19060e59927c service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:48.628391 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8f779cbb-b8af-48b3-b735-19060e59927c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:48.705539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f779cbb-b8af-48b3-b735-19060e59927c service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:48.710439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:48.711080 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:48.711357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:48.711626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] PATH_INFO: `/v3/users/152710e047354fbbb4aa5a0945e951e4` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:48.712851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=152710e047354fbbb4aa5a0945e951e4)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:48.714908 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:48.765302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '152710e047354fbbb4aa5a0945e951e4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:48.765994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '152710e047354fbbb4aa5a0945e951e4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:48.766590 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '152710e047354fbbb4aa5a0945e951e4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:48.778476 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '152710e047354fbbb4aa5a0945e951e4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:48.785759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '152710e047354fbbb4aa5a0945e951e4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:48.798336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1353/2712] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:48 2022] DELETE /identity/v3/users/152710e047354fbbb4aa5a0945e951e4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:48.802956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf1303-fb6a-4115-9d9a-94e1de51f753 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:48.804052 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf1303-fb6a-4115-9d9a-94e1de51f753 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:48.804371 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf1303-fb6a-4115-9d9a-94e1de51f753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:48.823252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-fedf1303-fb6a-4115-9d9a-94e1de51f753 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:48.824460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-fedf1303-fb6a-4115-9d9a-94e1de51f753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:48.889835 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1360/2713] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:48.900960 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:48.902431 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:48.966541 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201b4cd-cdb9-4468-979d-294ce4a51b57 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:48.970249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b035814-6a45-4d84-b2a1-a742b1bfc4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:48.970886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b035814-6a45-4d84-b2a1-a742b1bfc4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:48.971162 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b035814-6a45-4d84-b2a1-a742b1bfc4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:48.971429 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b035814-6a45-4d84-b2a1-a742b1bfc4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:48.981661 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b035814-6a45-4d84-b2a1-a742b1bfc4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:48.983094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b035814-6a45-4d84-b2a1-a742b1bfc4c3 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:48.993767 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1354/2714] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:49.296855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf1303-fb6a-4115-9d9a-94e1de51f753 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:49.298375 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-fedf1303-fb6a-4115-9d9a-94e1de51f753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:49.310381 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf1303-fb6a-4115-9d9a-94e1de51f753 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:49.315701 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:49.316362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:49.316639 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:49.317503 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] PATH_INFO: `/v3/projects/33a96d7c465f422aa6026281c3267da3` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:49.319649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33a96d7c465f422aa6026281c3267da3)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:49.321071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:49.373850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a96d7c465f422aa6026281c3267da3', 'request_id': 'req-d3a0887f-a44d-4150-96e7-03be73e52647'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:49.380000 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a96d7c465f422aa6026281c3267da3', 'request_id': 'req-d3a0887f-a44d-4150-96e7-03be73e52647'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:49.386084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a96d7c465f422aa6026281c3267da3', 'request_id': 'req-d3a0887f-a44d-4150-96e7-03be73e52647'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:49.399179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] The token cache is being invalidate because project 33a96d7c465f422aa6026281c3267da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:49.399465 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:49.400084 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:49.401585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1361/2715] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:49 2022] DELETE /identity/v3/projects/33a96d7c465f422aa6026281c3267da3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:49.405624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b035814-6a45-4d84-b2a1-a742b1bfc4c3 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:49.415466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b035814-6a45-4d84-b2a1-a742b1bfc4c3 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:49.433291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:49.473347 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c77e9-51ea-418e-acfc-d36f64358b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:49.473980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c77e9-51ea-418e-acfc-d36f64358b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:49.474265 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c77e9-51ea-418e-acfc-d36f64358b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:49.474585 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c77e9-51ea-418e-acfc-d36f64358b78 admin admin] PATH_INFO: `/v3/users/dded912f2b664f09b9f8d4a735ebb354` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:49.475819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c77e9-51ea-418e-acfc-d36f64358b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dded912f2b664f09b9f8d4a735ebb354)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:49.477764 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c77e9-51ea-418e-acfc-d36f64358b78 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:49.525216 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ea1c77e9-51ea-418e-acfc-d36f64358b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dded912f2b664f09b9f8d4a735ebb354'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:49.525919 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ea1c77e9-51ea-418e-acfc-d36f64358b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dded912f2b664f09b9f8d4a735ebb354'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:49.526517 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ea1c77e9-51ea-418e-acfc-d36f64358b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dded912f2b664f09b9f8d4a735ebb354'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:49.539538 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ea1c77e9-51ea-418e-acfc-d36f64358b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dded912f2b664f09b9f8d4a735ebb354'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:49.546899 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ea1c77e9-51ea-418e-acfc-d36f64358b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dded912f2b664f09b9f8d4a735ebb354'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:49.561274 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1355/2716] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:49 2022] DELETE /identity/v3/users/dded912f2b664f09b9f8d4a735ebb354 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:49.850411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:49.859362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:49.876698 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0887f-a44d-4150-96e7-03be73e52647 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:49.945588 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12893d2f-d5be-4dd6-88f8-c520479f220a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:49.946276 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12893d2f-d5be-4dd6-88f8-c520479f220a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:49.946660 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12893d2f-d5be-4dd6-88f8-c520479f220a admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:49.947060 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12893d2f-d5be-4dd6-88f8-c520479f220a admin admin] PATH_INFO: `/v3/projects/e6ee99f9409e44f0bf1484ac8f6197f2` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:49.948309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12893d2f-d5be-4dd6-88f8-c520479f220a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6ee99f9409e44f0bf1484ac8f6197f2)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:49.950404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12893d2f-d5be-4dd6-88f8-c520479f220a admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:49.980475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:49.998054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-12893d2f-d5be-4dd6-88f8-c520479f220a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6ee99f9409e44f0bf1484ac8f6197f2', 'request_id': 'req-12893d2f-d5be-4dd6-88f8-c520479f220a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:50.004013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-12893d2f-d5be-4dd6-88f8-c520479f220a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6ee99f9409e44f0bf1484ac8f6197f2', 'request_id': 'req-12893d2f-d5be-4dd6-88f8-c520479f220a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:50.009959 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-12893d2f-d5be-4dd6-88f8-c520479f220a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6ee99f9409e44f0bf1484ac8f6197f2', 'request_id': 'req-12893d2f-d5be-4dd6-88f8-c520479f220a'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:50.021894 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-12893d2f-d5be-4dd6-88f8-c520479f220a admin admin] The token cache is being invalidate because project e6ee99f9409e44f0bf1484ac8f6197f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:50.022126 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-12893d2f-d5be-4dd6-88f8-c520479f220a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:50.022799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-12893d2f-d5be-4dd6-88f8-c520479f220a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:50.024475 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1362/2717] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:49 2022] DELETE /identity/v3/projects/e6ee99f9409e44f0bf1484ac8f6197f2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:50.029327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269d2d9-4238-400e-9132-fffe194a51f5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.029838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269d2d9-4238-400e-9132-fffe194a51f5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.029962 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269d2d9-4238-400e-9132-fffe194a51f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.049742 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a269d2d9-4238-400e-9132-fffe194a51f5 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:50.050969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a269d2d9-4238-400e-9132-fffe194a51f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:50.124904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1356/2718] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:50.129379 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281da397-fbd2-4ffc-8799-45a568bfa690 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.129820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281da397-fbd2-4ffc-8799-45a568bfa690 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.130216 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281da397-fbd2-4ffc-8799-45a568bfa690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.153201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-281da397-fbd2-4ffc-8799-45a568bfa690 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:50.154512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-281da397-fbd2-4ffc-8799-45a568bfa690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:50.163265 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1363/2719] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:40:50.168053 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269d2d9-4238-400e-9132-fffe194a51f5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:50.180928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269d2d9-4238-400e-9132-fffe194a51f5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:50.186344 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ed5a6-2020-4cd8-b110-9c7c6520b9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:50.186974 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ed5a6-2020-4cd8-b110-9c7c6520b9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.187248 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ed5a6-2020-4cd8-b110-9c7c6520b9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.187906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ed5a6-2020-4cd8-b110-9c7c6520b9b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.188159 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ed5a6-2020-4cd8-b110-9c7c6520b9b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:40:50.188461 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ed5a6-2020-4cd8-b110-9c7c6520b9b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:50.190249 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ed5a6-2020-4cd8-b110-9c7c6520b9b7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:50.206625 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1357/2720] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:40:50.211543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281da397-fbd2-4ffc-8799-45a568bfa690 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:50.221616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281da397-fbd2-4ffc-8799-45a568bfa690 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:50.228219 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92032e-e30d-46f7-8a88-761164a5b198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:50.228863 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92032e-e30d-46f7-8a88-761164a5b198 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.229142 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92032e-e30d-46f7-8a88-761164a5b198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.229412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92032e-e30d-46f7-8a88-761164a5b198 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.230220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92032e-e30d-46f7-8a88-761164a5b198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:50.232307 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92032e-e30d-46f7-8a88-761164a5b198 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:50.279479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1364/2721] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:50 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:50.283304 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ed5a6-2020-4cd8-b110-9c7c6520b9b7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:50.295208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ed5a6-2020-4cd8-b110-9c7c6520b9b7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:50.322574 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:50.362966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd831d0-aeb7-4789-a7bd-3ae67377e657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:50.363639 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd831d0-aeb7-4789-a7bd-3ae67377e657 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.363920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd831d0-aeb7-4789-a7bd-3ae67377e657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.364321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd831d0-aeb7-4789-a7bd-3ae67377e657 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.364982 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd831d0-aeb7-4789-a7bd-3ae67377e657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:50.367822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd831d0-aeb7-4789-a7bd-3ae67377e657 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:50.388981 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1358/2722] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:50 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:50.393125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92032e-e30d-46f7-8a88-761164a5b198 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:50.405204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92032e-e30d-46f7-8a88-761164a5b198 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:50.410074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65833d43-8955-48ad-8d35-d9f24f4a7f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:50.410692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65833d43-8955-48ad-8d35-d9f24f4a7f35 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.410970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65833d43-8955-48ad-8d35-d9f24f4a7f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.411240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65833d43-8955-48ad-8d35-d9f24f4a7f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.411878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65833d43-8955-48ad-8d35-d9f24f4a7f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:50.414156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65833d43-8955-48ad-8d35-d9f24f4a7f35 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:50.424238 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1365/2723] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:50.428192 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd831d0-aeb7-4789-a7bd-3ae67377e657 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:50.441485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd831d0-aeb7-4789-a7bd-3ae67377e657 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:50.449362 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07a356-9539-46a2-924a-72a2b649358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:50.450086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07a356-9539-46a2-924a-72a2b649358f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.450583 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07a356-9539-46a2-924a-72a2b649358f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.450932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07a356-9539-46a2-924a-72a2b649358f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.451577 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07a356-9539-46a2-924a-72a2b649358f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:50.453203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07a356-9539-46a2-924a-72a2b649358f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:50.463826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1359/2724] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:50.468133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65833d43-8955-48ad-8d35-d9f24f4a7f35 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:50.479046 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65833d43-8955-48ad-8d35-d9f24f4a7f35 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:50.483699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686343d3-30df-4939-9620-83d6d5040d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:50.484344 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686343d3-30df-4939-9620-83d6d5040d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.484627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686343d3-30df-4939-9620-83d6d5040d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.484942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686343d3-30df-4939-9620-83d6d5040d8e admin admin] PATH_INFO: `/v3/projects/cb2f9d380e5a4442b4f373911f542a11/users/3dd4af69dbff498293f990e136f1e506/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.503555 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686343d3-30df-4939-9620-83d6d5040d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2f9d380e5a4442b4f373911f542a11, user_id=3dd4af69dbff498293f990e136f1e506, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:50.506241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686343d3-30df-4939-9620-83d6d5040d8e admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:50.518717 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1366/2725] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:50 2022] PUT /identity/v3/projects/cb2f9d380e5a4442b4f373911f542a11/users/3dd4af69dbff498293f990e136f1e506/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:50.531276 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ca93c-5455-47be-a9e3-2c831db9b91b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.531570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ca93c-5455-47be-a9e3-2c831db9b91b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.531850 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ca93c-5455-47be-a9e3-2c831db9b91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.560763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-dc1ca93c-5455-47be-a9e3-2c831db9b91b None None] MFA Rules not processed for user `3dd4af69dbff498293f990e136f1e506`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:50.562258 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-dc1ca93c-5455-47be-a9e3-2c831db9b91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:50.626345 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1360/2726] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:40:50.631031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686343d3-30df-4939-9620-83d6d5040d8e admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:50.694274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686343d3-30df-4939-9620-83d6d5040d8e admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:50.699124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbf42f-c23f-4adb-af0b-ca3c444f00c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:50.699719 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbf42f-c23f-4adb-af0b-ca3c444f00c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.699998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbf42f-c23f-4adb-af0b-ca3c444f00c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.700269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbf42f-c23f-4adb-af0b-ca3c444f00c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.700917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbf42f-c23f-4adb-af0b-ca3c444f00c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:50.702808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbf42f-c23f-4adb-af0b-ca3c444f00c0 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:50.729558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1367/2727] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:50 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:50.733687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ca93c-5455-47be-a9e3-2c831db9b91b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:50.745310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ca93c-5455-47be-a9e3-2c831db9b91b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:50.801609 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4373f81-c431-4d0d-9d10-d72943a91020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:50.802218 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4373f81-c431-4d0d-9d10-d72943a91020 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.802502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4373f81-c431-4d0d-9d10-d72943a91020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.802778 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4373f81-c431-4d0d-9d10-d72943a91020 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.803394 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4373f81-c431-4d0d-9d10-d72943a91020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:50.806013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4373f81-c431-4d0d-9d10-d72943a91020 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:50.827650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1361/2728] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:50 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:50.831881 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbf42f-c23f-4adb-af0b-ca3c444f00c0 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:50.842187 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbf42f-c23f-4adb-af0b-ca3c444f00c0 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:50.846678 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f77bd-4baa-4cea-8fee-69e00a233c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:50.847268 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f77bd-4baa-4cea-8fee-69e00a233c26 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.847541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f77bd-4baa-4cea-8fee-69e00a233c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.847810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f77bd-4baa-4cea-8fee-69e00a233c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.848434 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f77bd-4baa-4cea-8fee-69e00a233c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:50.850056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f77bd-4baa-4cea-8fee-69e00a233c26 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:50.861093 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1368/2729] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:50.864848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4373f81-c431-4d0d-9d10-d72943a91020 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:50.877330 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4373f81-c431-4d0d-9d10-d72943a91020 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:50.881998 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6449f-1261-4b2e-ba8d-ce4147e7b07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:50.882626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6449f-1261-4b2e-ba8d-ce4147e7b07b admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.883085 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6449f-1261-4b2e-ba8d-ce4147e7b07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.883199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6449f-1261-4b2e-ba8d-ce4147e7b07b admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.883834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6449f-1261-4b2e-ba8d-ce4147e7b07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:50.885533 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6449f-1261-4b2e-ba8d-ce4147e7b07b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:50.898972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1362/2730] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:50.900127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f77bd-4baa-4cea-8fee-69e00a233c26 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:50.910140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f77bd-4baa-4cea-8fee-69e00a233c26 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:50.915831 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756559f0-4c6f-4f48-9d5f-148e920fd2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:50.916468 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756559f0-4c6f-4f48-9d5f-148e920fd2b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.916742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756559f0-4c6f-4f48-9d5f-148e920fd2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.917038 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756559f0-4c6f-4f48-9d5f-148e920fd2b2 admin admin] PATH_INFO: `/v3/projects/5f222c2d8a494f50815f75c833d299aa/users/2b4490f3bc5f4666be7f48c75510558d/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.931837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756559f0-4c6f-4f48-9d5f-148e920fd2b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f222c2d8a494f50815f75c833d299aa, user_id=2b4490f3bc5f4666be7f48c75510558d, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:50.934490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756559f0-4c6f-4f48-9d5f-148e920fd2b2 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:50.943314 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:50.948374 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1369/2731] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:50 2022] PUT /identity/v3/projects/5f222c2d8a494f50815f75c833d299aa/users/2b4490f3bc5f4666be7f48c75510558d/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:50.958652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f256d46-c89a-4c3d-901a-7bc8cc1dda2f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:50.958931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f256d46-c89a-4c3d-901a-7bc8cc1dda2f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:50.959208 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f256d46-c89a-4c3d-901a-7bc8cc1dda2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:50.987074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7f256d46-c89a-4c3d-901a-7bc8cc1dda2f None None] MFA Rules not processed for user `2b4490f3bc5f4666be7f48c75510558d`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:50.988302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7f256d46-c89a-4c3d-901a-7bc8cc1dda2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:50.993083 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:51.054323 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1363/2732] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:51.058992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756559f0-4c6f-4f48-9d5f-148e920fd2b2 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:51.120068 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756559f0-4c6f-4f48-9d5f-148e920fd2b2 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:51.124675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd492c31-fa87-44e2-b762-cf3b095dab9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:51.125297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd492c31-fa87-44e2-b762-cf3b095dab9e admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:51.125576 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd492c31-fa87-44e2-b762-cf3b095dab9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:51.125846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd492c31-fa87-44e2-b762-cf3b095dab9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:51.126458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd492c31-fa87-44e2-b762-cf3b095dab9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:51.128342 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd492c31-fa87-44e2-b762-cf3b095dab9e admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:51.154933 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1370/2733] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:51 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:51.159111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f256d46-c89a-4c3d-901a-7bc8cc1dda2f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:51.170725 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f256d46-c89a-4c3d-901a-7bc8cc1dda2f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:51.227576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b342c-5c68-4026-bcda-ef63584a13ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:51.228222 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b342c-5c68-4026-bcda-ef63584a13ab admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:51.228508 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b342c-5c68-4026-bcda-ef63584a13ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:51.228807 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b342c-5c68-4026-bcda-ef63584a13ab admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:51.229436 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b342c-5c68-4026-bcda-ef63584a13ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:51.231387 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b342c-5c68-4026-bcda-ef63584a13ab admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:51.251699 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1364/2734] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:51 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:51.255869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd492c31-fa87-44e2-b762-cf3b095dab9e admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:51.266310 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd492c31-fa87-44e2-b762-cf3b095dab9e admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:51.270899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43cfda-b855-4be1-a5f2-bea4a04589d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:51.271484 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43cfda-b855-4be1-a5f2-bea4a04589d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:51.271761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43cfda-b855-4be1-a5f2-bea4a04589d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:51.272033 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43cfda-b855-4be1-a5f2-bea4a04589d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:51.272659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43cfda-b855-4be1-a5f2-bea4a04589d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:51.274769 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43cfda-b855-4be1-a5f2-bea4a04589d8 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:51.285322 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1371/2735] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:51.290221 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b342c-5c68-4026-bcda-ef63584a13ab admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:51.300513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b342c-5c68-4026-bcda-ef63584a13ab admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:51.305154 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2de89-3d99-47db-916d-78a442edad65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:51.305818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2de89-3d99-47db-916d-78a442edad65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:51.306101 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2de89-3d99-47db-916d-78a442edad65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:51.306382 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2de89-3d99-47db-916d-78a442edad65 admin admin] PATH_INFO: `/v3/projects/dfd58a75b6c84957944821da771a55d3/users/3fb35e8c660548a9b8de858de39fb39c/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:51.321479 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2de89-3d99-47db-916d-78a442edad65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd58a75b6c84957944821da771a55d3, user_id=3fb35e8c660548a9b8de858de39fb39c, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:51.324125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2de89-3d99-47db-916d-78a442edad65 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:51.339791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1365/2736] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:51 2022] PUT /identity/v3/projects/dfd58a75b6c84957944821da771a55d3/users/3fb35e8c660548a9b8de858de39fb39c/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:51.343758 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43cfda-b855-4be1-a5f2-bea4a04589d8 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:51.354486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43cfda-b855-4be1-a5f2-bea4a04589d8 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:51.412667 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a231c-2404-4696-9985-90dbd99b2acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:51.413317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a231c-2404-4696-9985-90dbd99b2acb admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:51.414256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a231c-2404-4696-9985-90dbd99b2acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:51.414537 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a231c-2404-4696-9985-90dbd99b2acb admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:51.415177 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a231c-2404-4696-9985-90dbd99b2acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:51.416777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a231c-2404-4696-9985-90dbd99b2acb admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:51.427333 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1372/2737] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:51.431693 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2de89-3d99-47db-916d-78a442edad65 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:51.445177 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2de89-3d99-47db-916d-78a442edad65 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:51.450359 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a6cd8-ea0b-485f-b64e-5807451d3124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:51.451013 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a6cd8-ea0b-485f-b64e-5807451d3124 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:51.451339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a6cd8-ea0b-485f-b64e-5807451d3124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:51.451615 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a6cd8-ea0b-485f-b64e-5807451d3124 admin admin] PATH_INFO: `/v3/projects/dfd58a75b6c84957944821da771a55d3/users/3fb35e8c660548a9b8de858de39fb39c/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:51.453559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a6cd8-ea0b-485f-b64e-5807451d3124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd58a75b6c84957944821da771a55d3, user_id=3fb35e8c660548a9b8de858de39fb39c, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:51.456175 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a6cd8-ea0b-485f-b64e-5807451d3124 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:51.466972 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1366/2738] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:51 2022] PUT /identity/v3/projects/dfd58a75b6c84957944821da771a55d3/users/3fb35e8c660548a9b8de858de39fb39c/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:51.478010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972b1f8-15f1-4816-a7a0-cd3cf7b42b30 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:51.478295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972b1f8-15f1-4816-a7a0-cd3cf7b42b30 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:51.478569 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972b1f8-15f1-4816-a7a0-cd3cf7b42b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:51.505446 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-9972b1f8-15f1-4816-a7a0-cd3cf7b42b30 None None] MFA Rules not processed for user `3fb35e8c660548a9b8de858de39fb39c`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:51.506584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9972b1f8-15f1-4816-a7a0-cd3cf7b42b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:51.565491 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:51.575200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1373/2739] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:51.587278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a6cd8-ea0b-485f-b64e-5807451d3124 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:51.589014 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c49a6cd8-ea0b-485f-b64e-5807451d3124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:51.645789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a6cd8-ea0b-485f-b64e-5807451d3124 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:51.650240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb6aa9-0cd4-491a-879e-1b86576f8cf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:51.650869 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb6aa9-0cd4-491a-879e-1b86576f8cf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:51.651145 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb6aa9-0cd4-491a-879e-1b86576f8cf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:51.651530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb6aa9-0cd4-491a-879e-1b86576f8cf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:51.660994 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb6aa9-0cd4-491a-879e-1b86576f8cf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:51.662439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb6aa9-0cd4-491a-879e-1b86576f8cf4 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:51.667721 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:51.672806 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1367/2740] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:51.713218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972b1f8-15f1-4816-a7a0-cd3cf7b42b30 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:51.714599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9972b1f8-15f1-4816-a7a0-cd3cf7b42b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:51.726683 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972b1f8-15f1-4816-a7a0-cd3cf7b42b30 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:51.730389 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8c253-48d9-4f61-a8a5-b7e1b0c55e3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:51.730987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8c253-48d9-4f61-a8a5-b7e1b0c55e3a service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:51.731285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8c253-48d9-4f61-a8a5-b7e1b0c55e3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:51.731554 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8c253-48d9-4f61-a8a5-b7e1b0c55e3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:51.788454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8c253-48d9-4f61-a8a5-b7e1b0c55e3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:51.789877 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8c253-48d9-4f61-a8a5-b7e1b0c55e3a service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:51.798629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1374/2741] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:55.728223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb6aa9-0cd4-491a-879e-1b86576f8cf4 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:55.791079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb6aa9-0cd4-491a-879e-1b86576f8cf4 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:55.795568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:55.796191 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:55.796465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:55.796736 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] PATH_INFO: `/v3/users/3dd4af69dbff498293f990e136f1e506` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:55.797868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd4af69dbff498293f990e136f1e506)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:55.799759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:55.851652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd4af69dbff498293f990e136f1e506'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:55.852303 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd4af69dbff498293f990e136f1e506'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:55.852946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd4af69dbff498293f990e136f1e506'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:55.864643 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd4af69dbff498293f990e136f1e506'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:55.871595 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd4af69dbff498293f990e136f1e506'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:55.884365 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1368/2742] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:55 2022] DELETE /identity/v3/users/3dd4af69dbff498293f990e136f1e506 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:55.888934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a7f6e-2821-4ba8-9393-6d75e165b015 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:55.889403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a7f6e-2821-4ba8-9393-6d75e165b015 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:55.889541 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a7f6e-2821-4ba8-9393-6d75e165b015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:55.909065 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-261a7f6e-2821-4ba8-9393-6d75e165b015 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:55.910207 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-261a7f6e-2821-4ba8-9393-6d75e165b015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:55.971002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1375/2743] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:55.980588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:55.982076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:56.042075 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35890a-a11d-4d7d-b371-e037175dac06 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:56.045690 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca133c87-2210-442c-b860-8b3c1d96db61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:56.046297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca133c87-2210-442c-b860-8b3c1d96db61 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:56.046697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca133c87-2210-442c-b860-8b3c1d96db61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:56.046975 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca133c87-2210-442c-b860-8b3c1d96db61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:56.056926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca133c87-2210-442c-b860-8b3c1d96db61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:56.058375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca133c87-2210-442c-b860-8b3c1d96db61 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:56.068287 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1369/2744] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:40:56.359632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a7f6e-2821-4ba8-9393-6d75e165b015 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:56.361226 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-261a7f6e-2821-4ba8-9393-6d75e165b015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:56.373922 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a7f6e-2821-4ba8-9393-6d75e165b015 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:56.378824 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:56.379527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:56.379883 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:56.380230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] PATH_INFO: `/v3/projects/cb2f9d380e5a4442b4f373911f542a11` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:56.382635 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb2f9d380e5a4442b4f373911f542a11)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:56.384632 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:56.437783 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb2f9d380e5a4442b4f373911f542a11', 'request_id': 'req-28f884af-f246-4464-ad96-cf798d70ec0d'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:56.444230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb2f9d380e5a4442b4f373911f542a11', 'request_id': 'req-28f884af-f246-4464-ad96-cf798d70ec0d'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:56.452159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb2f9d380e5a4442b4f373911f542a11', 'request_id': 'req-28f884af-f246-4464-ad96-cf798d70ec0d'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:56.466747 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] The token cache is being invalidate because project cb2f9d380e5a4442b4f373911f542a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:56.467040 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:56.467666 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:56.472361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1376/2745] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:56 2022] DELETE /identity/v3/projects/cb2f9d380e5a4442b4f373911f542a11 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:56.473716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca133c87-2210-442c-b860-8b3c1d96db61 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:56.484676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca133c87-2210-442c-b860-8b3c1d96db61 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:56.505146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:56.548256 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:56.548926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:56.549214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:56.549624 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] PATH_INFO: `/v3/users/2b4490f3bc5f4666be7f48c75510558d` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:56.550763 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4490f3bc5f4666be7f48c75510558d)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:56.552811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:56.601377 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4490f3bc5f4666be7f48c75510558d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:56.602039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4490f3bc5f4666be7f48c75510558d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:56.602663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4490f3bc5f4666be7f48c75510558d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:56.614591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4490f3bc5f4666be7f48c75510558d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:56.622588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4490f3bc5f4666be7f48c75510558d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:56.637909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1370/2746] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:56 2022] DELETE /identity/v3/users/2b4490f3bc5f4666be7f48c75510558d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:56.942895 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:56.944490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:56.959491 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f884af-f246-4464-ad96-cf798d70ec0d admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:56.973793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:57.019549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:57.020178 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:57.020457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:57.020727 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] PATH_INFO: `/v3/projects/5f222c2d8a494f50815f75c833d299aa` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:57.021858 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f222c2d8a494f50815f75c833d299aa)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:57.024264 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:57.069412 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f222c2d8a494f50815f75c833d299aa', 'request_id': 'req-9171ee87-1c8e-4a91-8380-88bfbcf5e417'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.075391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f222c2d8a494f50815f75c833d299aa', 'request_id': 'req-9171ee87-1c8e-4a91-8380-88bfbcf5e417'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.080892 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f222c2d8a494f50815f75c833d299aa', 'request_id': 'req-9171ee87-1c8e-4a91-8380-88bfbcf5e417'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.092876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] The token cache is being invalidate because project 5f222c2d8a494f50815f75c833d299aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:57.093168 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.093784 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.095266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1377/2747] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:56 2022] DELETE /identity/v3/projects/5f222c2d8a494f50815f75c833d299aa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:57.099310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:57.101438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:57.114032 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369b602-3e91-4855-b007-7e6f6660afe0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:57.170617 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde0dd1-57ea-4da3-a5ab-98cf4c62f668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:57.171237 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde0dd1-57ea-4da3-a5ab-98cf4c62f668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:57.171515 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde0dd1-57ea-4da3-a5ab-98cf4c62f668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:57.171782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde0dd1-57ea-4da3-a5ab-98cf4c62f668 admin admin] PATH_INFO: `/v3/users/3fb35e8c660548a9b8de858de39fb39c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:57.172904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde0dd1-57ea-4da3-a5ab-98cf4c62f668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fb35e8c660548a9b8de858de39fb39c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:57.174811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde0dd1-57ea-4da3-a5ab-98cf4c62f668 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:57.218611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:57.221196 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0cde0dd1-57ea-4da3-a5ab-98cf4c62f668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fb35e8c660548a9b8de858de39fb39c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.221932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0cde0dd1-57ea-4da3-a5ab-98cf4c62f668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fb35e8c660548a9b8de858de39fb39c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.222791 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0cde0dd1-57ea-4da3-a5ab-98cf4c62f668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fb35e8c660548a9b8de858de39fb39c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.236047 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0cde0dd1-57ea-4da3-a5ab-98cf4c62f668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fb35e8c660548a9b8de858de39fb39c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.243146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0cde0dd1-57ea-4da3-a5ab-98cf4c62f668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fb35e8c660548a9b8de858de39fb39c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.255525 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1371/2748] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:40:57 2022] DELETE /identity/v3/users/3fb35e8c660548a9b8de858de39fb39c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:57.525692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:57.527455 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:57.542324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171ee87-1c8e-4a91-8380-88bfbcf5e417 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:57.602532 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a588ed-1c0b-4027-b982-b410b1482252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:57.603171 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a588ed-1c0b-4027-b982-b410b1482252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:57.603450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a588ed-1c0b-4027-b982-b410b1482252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:57.603716 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a588ed-1c0b-4027-b982-b410b1482252 admin admin] PATH_INFO: `/v3/projects/dfd58a75b6c84957944821da771a55d3` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:57.604850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a588ed-1c0b-4027-b982-b410b1482252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfd58a75b6c84957944821da771a55d3)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:57.606735 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a588ed-1c0b-4027-b982-b410b1482252 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:57.664013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-26a588ed-1c0b-4027-b982-b410b1482252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfd58a75b6c84957944821da771a55d3', 'request_id': 'req-26a588ed-1c0b-4027-b982-b410b1482252'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.665287 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:57.672248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-26a588ed-1c0b-4027-b982-b410b1482252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfd58a75b6c84957944821da771a55d3', 'request_id': 'req-26a588ed-1c0b-4027-b982-b410b1482252'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.679942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-26a588ed-1c0b-4027-b982-b410b1482252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfd58a75b6c84957944821da771a55d3', 'request_id': 'req-26a588ed-1c0b-4027-b982-b410b1482252'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.691710 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-26a588ed-1c0b-4027-b982-b410b1482252 admin admin] The token cache is being invalidate because project dfd58a75b6c84957944821da771a55d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:40:57.691994 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-26a588ed-1c0b-4027-b982-b410b1482252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.692629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-26a588ed-1c0b-4027-b982-b410b1482252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:40:57.694111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1378/2749] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:40:57 2022] DELETE /identity/v3/projects/dfd58a75b6c84957944821da771a55d3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:57.700086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6273622-ff8c-4715-8d10-2421af8e4156 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:57.700373 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6273622-ff8c-4715-8d10-2421af8e4156 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:57.700646 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6273622-ff8c-4715-8d10-2421af8e4156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:57.719923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f6273622-ff8c-4715-8d10-2421af8e4156 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:57.721155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f6273622-ff8c-4715-8d10-2421af8e4156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:57.779900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1372/2750] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:57.789176 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd245204-ec45-4a3f-be94-0b8c2b03dfbb None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:57.789461 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd245204-ec45-4a3f-be94-0b8c2b03dfbb None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:57.789733 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd245204-ec45-4a3f-be94-0b8c2b03dfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:57.808724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-dd245204-ec45-4a3f-be94-0b8c2b03dfbb None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:57.809956 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-dd245204-ec45-4a3f-be94-0b8c2b03dfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:57.817980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1379/2751] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:57.822261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6273622-ff8c-4715-8d10-2421af8e4156 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:57.831909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6273622-ff8c-4715-8d10-2421af8e4156 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:57.837049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db9456-9c4a-4053-a5a9-aa81b29c0c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:57.837640 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db9456-9c4a-4053-a5a9-aa81b29c0c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:57.837921 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db9456-9c4a-4053-a5a9-aa81b29c0c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:57.838193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db9456-9c4a-4053-a5a9-aa81b29c0c28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:57.838834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db9456-9c4a-4053-a5a9-aa81b29c0c28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:40:57.839133 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db9456-9c4a-4053-a5a9-aa81b29c0c28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:57.840722 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db9456-9c4a-4053-a5a9-aa81b29c0c28 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:57.856561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1373/2752] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:40:57.857939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd245204-ec45-4a3f-be94-0b8c2b03dfbb None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:57.867556 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd245204-ec45-4a3f-be94-0b8c2b03dfbb None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:57.872211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff00fb7-2643-413b-b74a-306b6b022b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:57.872859 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff00fb7-2643-413b-b74a-306b6b022b23 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:57.873136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff00fb7-2643-413b-b74a-306b6b022b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:57.873405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff00fb7-2643-413b-b74a-306b6b022b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:57.874331 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff00fb7-2643-413b-b74a-306b6b022b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:57.876204 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff00fb7-2643-413b-b74a-306b6b022b23 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:57.905954 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1380/2753] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:40:57 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:57.909905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db9456-9c4a-4053-a5a9-aa81b29c0c28 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:57.920932 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db9456-9c4a-4053-a5a9-aa81b29c0c28 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:57.978017 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acaee5-ed01-43c1-a693-94c2e2a9d714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:57.978623 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acaee5-ed01-43c1-a693-94c2e2a9d714 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:57.978900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acaee5-ed01-43c1-a693-94c2e2a9d714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:57.979171 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acaee5-ed01-43c1-a693-94c2e2a9d714 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:57.979792 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acaee5-ed01-43c1-a693-94c2e2a9d714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:57.981748 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acaee5-ed01-43c1-a693-94c2e2a9d714 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:58.003152 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1374/2754] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:40:57 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:40:58.007352 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff00fb7-2643-413b-b74a-306b6b022b23 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:58.017478 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff00fb7-2643-413b-b74a-306b6b022b23 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:58.021917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693415c-5917-4df3-b5e5-af434cd65ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:58.023490 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693415c-5917-4df3-b5e5-af434cd65ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:58.023808 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693415c-5917-4df3-b5e5-af434cd65ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:58.024085 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693415c-5917-4df3-b5e5-af434cd65ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:58.024735 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693415c-5917-4df3-b5e5-af434cd65ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:58.026359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693415c-5917-4df3-b5e5-af434cd65ac5 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:58.036428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1381/2755] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:40:58.041054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acaee5-ed01-43c1-a693-94c2e2a9d714 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:58.051279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acaee5-ed01-43c1-a693-94c2e2a9d714 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:58.056006 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c211d-c711-4c98-af9e-611c3d0691c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:58.056599 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c211d-c711-4c98-af9e-611c3d0691c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:58.056913 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c211d-c711-4c98-af9e-611c3d0691c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:58.057187 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c211d-c711-4c98-af9e-611c3d0691c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:58.057818 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c211d-c711-4c98-af9e-611c3d0691c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:58.059393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c211d-c711-4c98-af9e-611c3d0691c7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:58.069996 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1375/2756] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:40:58.074005 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693415c-5917-4df3-b5e5-af434cd65ac5 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:58.084080 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693415c-5917-4df3-b5e5-af434cd65ac5 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:58.089259 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdfa19-ac7e-4782-9e02-c1f43be3ea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:58.089905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdfa19-ac7e-4782-9e02-c1f43be3ea07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:58.090182 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdfa19-ac7e-4782-9e02-c1f43be3ea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:58.090457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdfa19-ac7e-4782-9e02-c1f43be3ea07 admin admin] PATH_INFO: `/v3/projects/fa70bd211f7948928b871355151b6259/users/032565eefb8e4913a1223d9e5f1ef915/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:58.105610 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdfa19-ac7e-4782-9e02-c1f43be3ea07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa70bd211f7948928b871355151b6259, user_id=032565eefb8e4913a1223d9e5f1ef915, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:58.108443 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdfa19-ac7e-4782-9e02-c1f43be3ea07 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:58.121107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1382/2757] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:40:58 2022] PUT /identity/v3/projects/fa70bd211f7948928b871355151b6259/users/032565eefb8e4913a1223d9e5f1ef915/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:40:58.132569 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4492d-9af5-481b-8b1d-400f4161929a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:58.133079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4492d-9af5-481b-8b1d-400f4161929a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:58.133179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4492d-9af5-481b-8b1d-400f4161929a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:58.160977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bec4492d-9af5-481b-8b1d-400f4161929a None None] MFA Rules not processed for user `032565eefb8e4913a1223d9e5f1ef915`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:40:58.162172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bec4492d-9af5-481b-8b1d-400f4161929a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:58.175257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:40:58.230122 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1376/2758] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:40:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:40:58.240902 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdfa19-ac7e-4782-9e02-c1f43be3ea07 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:40:58.242662 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-95bdfa19-ac7e-4782-9e02-c1f43be3ea07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:40:58.299232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdfa19-ac7e-4782-9e02-c1f43be3ea07 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:40:58.302749 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8555cfe-e042-4d69-9b9a-8665fa40c7eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:40:58.303338 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8555cfe-e042-4d69-9b9a-8665fa40c7eb service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:40:58.303660 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8555cfe-e042-4d69-9b9a-8665fa40c7eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:40:58.303931 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8555cfe-e042-4d69-9b9a-8665fa40c7eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:40:58.312343 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8555cfe-e042-4d69-9b9a-8665fa40c7eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:40:58.313793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8555cfe-e042-4d69-9b9a-8665fa40c7eb service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:40:58.323232 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1383/2759] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:40:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:02.345789 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315381c6-4afe-41d6-9db7-d0e820432a60 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:02.346237 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315381c6-4afe-41d6-9db7-d0e820432a60 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:02.346631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315381c6-4afe-41d6-9db7-d0e820432a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:02.378696 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-315381c6-4afe-41d6-9db7-d0e820432a60 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:02.380002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-315381c6-4afe-41d6-9db7-d0e820432a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:02.438890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1377/2760] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:02.471830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8555cfe-e042-4d69-9b9a-8665fa40c7eb service cinder] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:02.488155 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315381c6-4afe-41d6-9db7-d0e820432a60 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:02.489716 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-315381c6-4afe-41d6-9db7-d0e820432a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:02.549435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8555cfe-e042-4d69-9b9a-8665fa40c7eb service cinder] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:02.555477 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:02.556137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:02.556472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:02.556895 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] PATH_INFO: `/v3/users/032565eefb8e4913a1223d9e5f1ef915` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:02.558713 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=032565eefb8e4913a1223d9e5f1ef915)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:02.561003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:02.584246 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315381c6-4afe-41d6-9db7-d0e820432a60 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:02.588127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:02.590688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b4ea8-a2f0-40c1-b609-edfe7094d083 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:02.591319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b4ea8-a2f0-40c1-b609-edfe7094d083 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:02.591650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b4ea8-a2f0-40c1-b609-edfe7094d083 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:02.591877 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b4ea8-a2f0-40c1-b609-edfe7094d083 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:02.601847 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b4ea8-a2f0-40c1-b609-edfe7094d083 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:02.603909 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b4ea8-a2f0-40c1-b609-edfe7094d083 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:02.614054 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1378/2761] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:41:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:41:02.621760 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '032565eefb8e4913a1223d9e5f1ef915'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:02.622497 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '032565eefb8e4913a1223d9e5f1ef915'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:02.623174 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '032565eefb8e4913a1223d9e5f1ef915'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:02.633263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '032565eefb8e4913a1223d9e5f1ef915'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:02.648721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '032565eefb8e4913a1223d9e5f1ef915'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:02.673597 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1384/2762] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:41:02 2022] DELETE /identity/v3/users/032565eefb8e4913a1223d9e5f1ef915 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:02.678120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9c2d8-1ca5-4279-9624-a4add4acd4b0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:02.679293 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9c2d8-1ca5-4279-9624-a4add4acd4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:02.679733 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9c2d8-1ca5-4279-9624-a4add4acd4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:02.699683 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-42b9c2d8-1ca5-4279-9624-a4add4acd4b0 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:02.700941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-42b9c2d8-1ca5-4279-9624-a4add4acd4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:02.750332 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:02.775626 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1379/2763] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:02.785761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:02.787387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:02.860736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed083f-2cc6-4bf6-8da7-2abd24747714 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:02.867435 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6084c-e6ef-4e22-ba84-b700f42add1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:02.868184 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6084c-e6ef-4e22-ba84-b700f42add1a service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:02.868548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6084c-e6ef-4e22-ba84-b700f42add1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:02.868913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6084c-e6ef-4e22-ba84-b700f42add1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:02.878636 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6084c-e6ef-4e22-ba84-b700f42add1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:02.880163 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6084c-e6ef-4e22-ba84-b700f42add1a service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:02.890225 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1385/2764] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:03.168931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9c2d8-1ca5-4279-9624-a4add4acd4b0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:03.170611 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-42b9c2d8-1ca5-4279-9624-a4add4acd4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:03.183068 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9c2d8-1ca5-4279-9624-a4add4acd4b0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:03.188164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b607b8-9795-429b-ab26-3505233c3a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:03.188828 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b607b8-9795-429b-ab26-3505233c3a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.189230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b607b8-9795-429b-ab26-3505233c3a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.189401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b607b8-9795-429b-ab26-3505233c3a68 admin admin] PATH_INFO: `/v3/projects/fa70bd211f7948928b871355151b6259` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.190557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b607b8-9795-429b-ab26-3505233c3a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa70bd211f7948928b871355151b6259)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:03.192470 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b607b8-9795-429b-ab26-3505233c3a68 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:03.197290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271ad1d-9de5-4b33-ac7d-d420e21bb659 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.197582 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271ad1d-9de5-4b33-ac7d-d420e21bb659 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.197857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271ad1d-9de5-4b33-ac7d-d420e21bb659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.218927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d271ad1d-9de5-4b33-ac7d-d420e21bb659 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:03.220115 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d271ad1d-9de5-4b33-ac7d-d420e21bb659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:03.246039 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-08b607b8-9795-429b-ab26-3505233c3a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa70bd211f7948928b871355151b6259', 'request_id': 'req-08b607b8-9795-429b-ab26-3505233c3a68'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:03.251927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-08b607b8-9795-429b-ab26-3505233c3a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa70bd211f7948928b871355151b6259', 'request_id': 'req-08b607b8-9795-429b-ab26-3505233c3a68'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:03.258339 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-08b607b8-9795-429b-ab26-3505233c3a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa70bd211f7948928b871355151b6259', 'request_id': 'req-08b607b8-9795-429b-ab26-3505233c3a68'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:03.272852 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-08b607b8-9795-429b-ab26-3505233c3a68 admin admin] The token cache is being invalidate because project fa70bd211f7948928b871355151b6259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:41:03.273146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-08b607b8-9795-429b-ab26-3505233c3a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:03.274217 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-08b607b8-9795-429b-ab26-3505233c3a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:03.276028 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1380/2765] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:41:03 2022] DELETE /identity/v3/projects/fa70bd211f7948928b871355151b6259 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:03.278131 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1386/2766] 10.222.0.55 () {58 vars in 967 bytes} [Fri Aug 19 16:41:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:41:03.280674 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052a949-5174-4d8f-8f35-06fad952b4d8 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.280930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052a949-5174-4d8f-8f35-06fad952b4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.281209 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052a949-5174-4d8f-8f35-06fad952b4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.291270 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271ad1d-9de5-4b33-ac7d-d420e21bb659 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:03.292697 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d271ad1d-9de5-4b33-ac7d-d420e21bb659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:03.301240 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5052a949-5174-4d8f-8f35-06fad952b4d8 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:03.302439 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5052a949-5174-4d8f-8f35-06fad952b4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:03.339468 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:03.366500 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1381/2767] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:41:03.367543 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271ad1d-9de5-4b33-ac7d-d420e21bb659 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:03.435977 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07f86b-3045-4b05-876d-e7b39f5c4c7f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.436277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07f86b-3045-4b05-876d-e7b39f5c4c7f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.436561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07f86b-3045-4b05-876d-e7b39f5c4c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.457450 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4a07f86b-3045-4b05-876d-e7b39f5c4c7f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:03.458200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbe161-4fa0-4836-9e13-15f5f57ff504 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:03.458920 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4a07f86b-3045-4b05-876d-e7b39f5c4c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:03.459337 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbe161-4fa0-4836-9e13-15f5f57ff504 service swift] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.460256 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbe161-4fa0-4836-9e13-15f5f57ff504 service swift] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.460904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbe161-4fa0-4836-9e13-15f5f57ff504 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.470864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1382/2768] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:03.474570 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07f86b-3045-4b05-876d-e7b39f5c4c7f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:03.484015 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07f86b-3045-4b05-876d-e7b39f5c4c7f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:03.492356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195b0d9-d24f-4551-8d4b-ffea0dcd3837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:03.493048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195b0d9-d24f-4551-8d4b-ffea0dcd3837 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.493272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195b0d9-d24f-4551-8d4b-ffea0dcd3837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.493552 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195b0d9-d24f-4551-8d4b-ffea0dcd3837 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.494206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195b0d9-d24f-4551-8d4b-ffea0dcd3837 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:41:03.494514 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195b0d9-d24f-4551-8d4b-ffea0dcd3837 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:03.496113 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195b0d9-d24f-4551-8d4b-ffea0dcd3837 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:03.513889 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1383/2769] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:41:03.515040 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195b0d9-d24f-4551-8d4b-ffea0dcd3837 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:03.527726 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195b0d9-d24f-4551-8d4b-ffea0dcd3837 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:03.533410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb30486-d98e-4814-9e00-3f86b81904a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:03.534195 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbe161-4fa0-4836-9e13-15f5f57ff504 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:03.534656 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb30486-d98e-4814-9e00-3f86b81904a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.534794 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb30486-d98e-4814-9e00-3f86b81904a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.535161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb30486-d98e-4814-9e00-3f86b81904a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.535861 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbe161-4fa0-4836-9e13-15f5f57ff504 service swift] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:03.536022 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb30486-d98e-4814-9e00-3f86b81904a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:03.538619 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb30486-d98e-4814-9e00-3f86b81904a4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:03.545799 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1387/2770] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:41:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 19 16:41:03.571948 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1384/2771] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:41:03 2022] POST /identity/v3/projects => generated 397 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:41:03.576237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbe161-4fa0-4836-9e13-15f5f57ff504 service swift] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:03.589170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbe161-4fa0-4836-9e13-15f5f57ff504 service swift] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:03.651230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72487f-6856-460b-9520-365af2b81fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:03.651834 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72487f-6856-460b-9520-365af2b81fab admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.652127 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72487f-6856-460b-9520-365af2b81fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.652400 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72487f-6856-460b-9520-365af2b81fab admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.653072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72487f-6856-460b-9520-365af2b81fab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:03.654990 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72487f-6856-460b-9520-365af2b81fab admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:03.678736 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1388/2772] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:41:03 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:41:03.682797 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb30486-d98e-4814-9e00-3f86b81904a4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:03.694826 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb30486-d98e-4814-9e00-3f86b81904a4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:03.699587 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29cc79-12d1-4003-a705-59d1d81502e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:03.700261 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29cc79-12d1-4003-a705-59d1d81502e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.700616 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29cc79-12d1-4003-a705-59d1d81502e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.700993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29cc79-12d1-4003-a705-59d1d81502e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.701906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be29cc79-12d1-4003-a705-59d1d81502e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:03.703513 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be29cc79-12d1-4003-a705-59d1d81502e2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:03.716189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1385/2773] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:41:03.719970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72487f-6856-460b-9520-365af2b81fab admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:03.733387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72487f-6856-460b-9520-365af2b81fab admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:03.738249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64862524-bab0-4fc7-ac23-74023c04a98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:03.738938 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64862524-bab0-4fc7-ac23-74023c04a98b admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.739305 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64862524-bab0-4fc7-ac23-74023c04a98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.739622 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64862524-bab0-4fc7-ac23-74023c04a98b admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.740320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64862524-bab0-4fc7-ac23-74023c04a98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:03.742269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64862524-bab0-4fc7-ac23-74023c04a98b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:03.752830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1389/2774] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:41:03.757697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29cc79-12d1-4003-a705-59d1d81502e2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:03.768406 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29cc79-12d1-4003-a705-59d1d81502e2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:03.774298 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244f6ec-a2bc-4278-b5c8-de898542fcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:03.774955 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244f6ec-a2bc-4278-b5c8-de898542fcb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.775326 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244f6ec-a2bc-4278-b5c8-de898542fcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.775492 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244f6ec-a2bc-4278-b5c8-de898542fcb6 admin admin] PATH_INFO: `/v3/projects/095d7bf0a7524b5a8033663f1c3e7988/users/88a674908fda48b5b2ed89d596e55be7/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.785724 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb51384-2e29-4305-b7c0-e17283a84ab0 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.786013 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb51384-2e29-4305-b7c0-e17283a84ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.786286 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb51384-2e29-4305-b7c0-e17283a84ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.788418 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:03.794434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244f6ec-a2bc-4278-b5c8-de898542fcb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095d7bf0a7524b5a8033663f1c3e7988, user_id=88a674908fda48b5b2ed89d596e55be7, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:03.798939 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244f6ec-a2bc-4278-b5c8-de898542fcb6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:03.802550 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:03.809816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-efb51384-2e29-4305-b7c0-e17283a84ab0 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:03.811608 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-efb51384-2e29-4305-b7c0-e17283a84ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:03.819694 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1386/2775] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:41:03 2022] PUT /identity/v3/projects/095d7bf0a7524b5a8033663f1c3e7988/users/88a674908fda48b5b2ed89d596e55be7/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:41:03.829911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031630c-cd14-4c48-b1d9-1c281811ae00 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.830254 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031630c-cd14-4c48-b1d9-1c281811ae00 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.830531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031630c-cd14-4c48-b1d9-1c281811ae00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.859108 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8031630c-cd14-4c48-b1d9-1c281811ae00 None None] MFA Rules not processed for user `88a674908fda48b5b2ed89d596e55be7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:03.860323 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8031630c-cd14-4c48-b1d9-1c281811ae00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:03.875245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1390/2776] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Aug 19 16:41:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:41:03.888114 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb51384-2e29-4305-b7c0-e17283a84ab0 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:03.930213 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1387/2777] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:03 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:41:03.943211 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031630c-cd14-4c48-b1d9-1c281811ae00 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:03.945268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8031630c-cd14-4c48-b1d9-1c281811ae00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:03.962021 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb51384-2e29-4305-b7c0-e17283a84ab0 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:03.967705 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908a1af-7ccf-4ea1-96d7-c08d0992d71f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:03.968251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908a1af-7ccf-4ea1-96d7-c08d0992d71f service swift] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:03.968527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908a1af-7ccf-4ea1-96d7-c08d0992d71f service swift] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:03.968820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908a1af-7ccf-4ea1-96d7-c08d0992d71f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:03.977750 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908a1af-7ccf-4ea1-96d7-c08d0992d71f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:03.979558 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908a1af-7ccf-4ea1-96d7-c08d0992d71f service swift] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:03.990111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1391/2778] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:41:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 19 16:41:04.005834 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031630c-cd14-4c48-b1d9-1c281811ae00 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:04.009991 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cce19-f79d-4713-9115-b7a715306657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:04.010601 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cce19-f79d-4713-9115-b7a715306657 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:04.010879 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cce19-f79d-4713-9115-b7a715306657 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:04.011147 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cce19-f79d-4713-9115-b7a715306657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:04.019992 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cce19-f79d-4713-9115-b7a715306657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:04.021465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cce19-f79d-4713-9115-b7a715306657 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:04.030890 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1388/2779] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:03 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:04.353124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9c39d-dd65-46e3-b796-fc45ea63c697 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:04.353507 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9c39d-dd65-46e3-b796-fc45ea63c697 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:04.353952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9c39d-dd65-46e3-b796-fc45ea63c697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:04.374792 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-fae9c39d-dd65-46e3-b796-fc45ea63c697 None None] MFA Rules not processed for user `88694a50b9f04487a521ba6df110d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:04.376027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-fae9c39d-dd65-46e3-b796-fc45ea63c697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:04.383423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1392/2780] 10.222.0.55 () {58 vars in 967 bytes} [Fri Aug 19 16:41:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:04.396001 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cce19-f79d-4713-9115-b7a715306657 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:04.408010 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cce19-f79d-4713-9115-b7a715306657 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:04.413956 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110608b-0737-4528-8478-7c24595d7ab5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c755f9538e984bc3877112896595e158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:04.415410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110608b-0737-4528-8478-7c24595d7ab5 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:04.415410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110608b-0737-4528-8478-7c24595d7ab5 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:04.415410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110608b-0737-4528-8478-7c24595d7ab5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:04.424000 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110608b-0737-4528-8478-7c24595d7ab5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:04.425641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110608b-0737-4528-8478-7c24595d7ab5 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:04.440670 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1389/2781] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Aug 19 16:41:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 16:41:05.722620 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9c39d-dd65-46e3-b796-fc45ea63c697 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:05.731148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-fae9c39d-dd65-46e3-b796-fc45ea63c697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:05.798641 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9c39d-dd65-46e3-b796-fc45ea63c697 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:05.803752 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:05.804453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:05.804737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:05.805037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] PATH_INFO: `/v3/users/0f5756987a42401c816a24e3173f40ae` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:05.806241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f5756987a42401c816a24e3173f40ae)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:05.808134 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:05.848822 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:05.859495 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f5756987a42401c816a24e3173f40ae'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:05.860157 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f5756987a42401c816a24e3173f40ae'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:05.860882 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f5756987a42401c816a24e3173f40ae'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:05.868681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f5756987a42401c816a24e3173f40ae'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:05.880237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f5756987a42401c816a24e3173f40ae'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:05.894015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1393/2782] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:41:05 2022] DELETE /identity/v3/users/0f5756987a42401c816a24e3173f40ae => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:05.898705 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96db7b7-0555-4ae8-8da2-9eb1412114e5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:05.899314 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96db7b7-0555-4ae8-8da2-9eb1412114e5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:05.899314 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96db7b7-0555-4ae8-8da2-9eb1412114e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:05.918825 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a96db7b7-0555-4ae8-8da2-9eb1412114e5 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:05.920074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a96db7b7-0555-4ae8-8da2-9eb1412114e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:05.979532 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1390/2783] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:41:05.989317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:05.991429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:06.048248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8ee9f-3550-4e6f-a1fa-1a27f0616470 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:06.051965 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b212d3-358c-4e95-93f5-9a2dd5992273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:06.052499 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b212d3-358c-4e95-93f5-9a2dd5992273 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.052805 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b212d3-358c-4e95-93f5-9a2dd5992273 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.053212 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b212d3-358c-4e95-93f5-9a2dd5992273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.062402 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b212d3-358c-4e95-93f5-9a2dd5992273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:06.063809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b212d3-358c-4e95-93f5-9a2dd5992273 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:06.073255 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1394/2784] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:06.342653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96db7b7-0555-4ae8-8da2-9eb1412114e5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:06.344097 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a96db7b7-0555-4ae8-8da2-9eb1412114e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:06.358287 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96db7b7-0555-4ae8-8da2-9eb1412114e5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:06.362950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:06.364227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.364518 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.364894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e admin admin] PATH_INFO: `/v3/projects/17acb86dbdc24b92b42c368e3cc1af28` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.366130 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=17acb86dbdc24b92b42c368e3cc1af28)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:06.368022 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:06.370575 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:06.417433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17acb86dbdc24b92b42c368e3cc1af28', 'request_id': 'req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:06.422837 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17acb86dbdc24b92b42c368e3cc1af28', 'request_id': 'req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:06.429279 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17acb86dbdc24b92b42c368e3cc1af28', 'request_id': 'req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:06.442523 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e admin admin] The token cache is being invalidate because project 17acb86dbdc24b92b42c368e3cc1af28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:41:06.442814 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:06.443463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7f6db2f9-42f9-43e3-baab-d2c3648d4a3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:06.445492 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1391/2785] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:41:06 2022] DELETE /identity/v3/projects/17acb86dbdc24b92b42c368e3cc1af28 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:06.450457 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1e473-6464-4b19-8a26-5450408dee85 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.450746 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1e473-6464-4b19-8a26-5450408dee85 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.451022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1e473-6464-4b19-8a26-5450408dee85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.470430 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-aab1e473-6464-4b19-8a26-5450408dee85 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:06.471624 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-aab1e473-6464-4b19-8a26-5450408dee85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:06.529516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1395/2786] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:06.537779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835e207-96bb-4252-a29c-8fb93c67e157 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.538070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835e207-96bb-4252-a29c-8fb93c67e157 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.538344 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835e207-96bb-4252-a29c-8fb93c67e157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.559375 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7835e207-96bb-4252-a29c-8fb93c67e157 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:06.559534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1e473-6464-4b19-8a26-5450408dee85 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:06.561401 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7835e207-96bb-4252-a29c-8fb93c67e157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:06.561989 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-aab1e473-6464-4b19-8a26-5450408dee85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:06.569819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1392/2787] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:06.573759 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835e207-96bb-4252-a29c-8fb93c67e157 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:06.580233 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1e473-6464-4b19-8a26-5450408dee85 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:06.585161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835e207-96bb-4252-a29c-8fb93c67e157 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:06.585629 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de822ca1-7553-4802-84dc-f02c955e069b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:06.586200 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de822ca1-7553-4802-84dc-f02c955e069b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.586480 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de822ca1-7553-4802-84dc-f02c955e069b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.586756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de822ca1-7553-4802-84dc-f02c955e069b admin admin] PATH_INFO: `/v3/users/5f0dd23f0f0b4891a9933cbe8a4861f3` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.587876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de822ca1-7553-4802-84dc-f02c955e069b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0dd23f0f0b4891a9933cbe8a4861f3)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:06.589846 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de822ca1-7553-4802-84dc-f02c955e069b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:06.590334 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c9a00-f6cf-4eb0-9f6e-224c9d4a158d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:06.590928 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c9a00-f6cf-4eb0-9f6e-224c9d4a158d admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.591205 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c9a00-f6cf-4eb0-9f6e-224c9d4a158d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.591474 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c9a00-f6cf-4eb0-9f6e-224c9d4a158d admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.592106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c9a00-f6cf-4eb0-9f6e-224c9d4a158d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:41:06.592408 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c9a00-f6cf-4eb0-9f6e-224c9d4a158d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:06.594020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c9a00-f6cf-4eb0-9f6e-224c9d4a158d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:06.606588 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1393/2788] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 16:41:06.610815 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c9a00-f6cf-4eb0-9f6e-224c9d4a158d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:06.622094 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c9a00-f6cf-4eb0-9f6e-224c9d4a158d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:06.627633 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa407ad-1ed2-477e-9b16-9177beef15d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:06.628228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa407ad-1ed2-477e-9b16-9177beef15d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.628504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa407ad-1ed2-477e-9b16-9177beef15d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.628820 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa407ad-1ed2-477e-9b16-9177beef15d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.629462 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa407ad-1ed2-477e-9b16-9177beef15d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:06.631880 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa407ad-1ed2-477e-9b16-9177beef15d2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:06.647486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-de822ca1-7553-4802-84dc-f02c955e069b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0dd23f0f0b4891a9933cbe8a4861f3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:06.648246 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-de822ca1-7553-4802-84dc-f02c955e069b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0dd23f0f0b4891a9933cbe8a4861f3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:06.649027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-de822ca1-7553-4802-84dc-f02c955e069b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0dd23f0f0b4891a9933cbe8a4861f3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:06.655801 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:06.658269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-de822ca1-7553-4802-84dc-f02c955e069b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0dd23f0f0b4891a9933cbe8a4861f3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:06.661092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1394/2789] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:41:06 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:41:06.665383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa407ad-1ed2-477e-9b16-9177beef15d2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:06.667186 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4fa407ad-1ed2-477e-9b16-9177beef15d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:06.673328 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-de822ca1-7553-4802-84dc-f02c955e069b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0dd23f0f0b4891a9933cbe8a4861f3'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:06.686540 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1396/2790] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:41:06 2022] DELETE /identity/v3/users/5f0dd23f0f0b4891a9933cbe8a4861f3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:41:06.690849 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89d0a4-fce5-446c-af21-17d88eef4f06 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.691133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89d0a4-fce5-446c-af21-17d88eef4f06 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.691408 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89d0a4-fce5-446c-af21-17d88eef4f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.710656 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-6f89d0a4-fce5-446c-af21-17d88eef4f06 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:06.711851 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6f89d0a4-fce5-446c-af21-17d88eef4f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:06.737311 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa407ad-1ed2-477e-9b16-9177beef15d2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:06.772649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1397/2791] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:41:06.782589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89d0a4-fce5-446c-af21-17d88eef4f06 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:06.784095 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-6f89d0a4-fce5-446c-af21-17d88eef4f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:06.795150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:06.802399 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ad514-70bf-483c-988e-249050a2a1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:06.804565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ad514-70bf-483c-988e-249050a2a1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.805024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ad514-70bf-483c-988e-249050a2a1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.805865 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ad514-70bf-483c-988e-249050a2a1e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.806664 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ad514-70bf-483c-988e-249050a2a1e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:06.809065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ad514-70bf-483c-988e-249050a2a1e1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:06.833677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1395/2792] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:41:06 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:41:06.835386 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ad514-70bf-483c-988e-249050a2a1e1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:06.845168 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ad514-70bf-483c-988e-249050a2a1e1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:06.846035 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89d0a4-fce5-446c-af21-17d88eef4f06 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:06.849846 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877a4c8-686d-45d8-9c5c-1fc535e3ca2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:06.850017 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d06ff-0617-4618-814a-7f4b393a5f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:06.850596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877a4c8-686d-45d8-9c5c-1fc535e3ca2b admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.850721 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d06ff-0617-4618-814a-7f4b393a5f49 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.850878 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877a4c8-686d-45d8-9c5c-1fc535e3ca2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.851025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d06ff-0617-4618-814a-7f4b393a5f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.851140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877a4c8-686d-45d8-9c5c-1fc535e3ca2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.851236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d06ff-0617-4618-814a-7f4b393a5f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.851770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877a4c8-686d-45d8-9c5c-1fc535e3ca2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:06.853397 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877a4c8-686d-45d8-9c5c-1fc535e3ca2b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:06.861022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d06ff-0617-4618-814a-7f4b393a5f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:06.862797 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d06ff-0617-4618-814a-7f4b393a5f49 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:06.866874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1396/2793] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:41:06.867861 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877a4c8-686d-45d8-9c5c-1fc535e3ca2b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:06.874295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1398/2794] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:41:06.877684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877a4c8-686d-45d8-9c5c-1fc535e3ca2b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:06.883076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68d4f8-6f1c-4fe0-aa52-e9a388eb02c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:06.883628 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68d4f8-6f1c-4fe0-aa52-e9a388eb02c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.883904 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68d4f8-6f1c-4fe0-aa52-e9a388eb02c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.884178 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68d4f8-6f1c-4fe0-aa52-e9a388eb02c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.884914 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68d4f8-6f1c-4fe0-aa52-e9a388eb02c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:06.886511 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68d4f8-6f1c-4fe0-aa52-e9a388eb02c8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:06.900484 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1397/2795] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:41:06.906591 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d06ff-0617-4618-814a-7f4b393a5f49 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:06.919062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d06ff-0617-4618-814a-7f4b393a5f49 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:06.923942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37f154-9611-4454-9210-acacb0181c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:06.924872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37f154-9611-4454-9210-acacb0181c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.925149 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37f154-9611-4454-9210-acacb0181c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.925454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37f154-9611-4454-9210-acacb0181c80 admin admin] PATH_INFO: `/v3/projects/f9697dfddc5f4b4fa66d39e2e87e1bdc/users/41db22367b0b454980e076378eb019be/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:06.942030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37f154-9611-4454-9210-acacb0181c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9697dfddc5f4b4fa66d39e2e87e1bdc, user_id=41db22367b0b454980e076378eb019be, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:06.945327 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37f154-9611-4454-9210-acacb0181c80 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:06.961651 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1399/2796] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:41:06 2022] PUT /identity/v3/projects/f9697dfddc5f4b4fa66d39e2e87e1bdc/users/41db22367b0b454980e076378eb019be/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:06.973559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e42780-f554-4797-a989-d526df665080 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:06.973848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e42780-f554-4797-a989-d526df665080 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:06.974239 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e42780-f554-4797-a989-d526df665080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.008984 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e5e42780-f554-4797-a989-d526df665080 None None] MFA Rules not processed for user `41db22367b0b454980e076378eb019be`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:07.010238 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e5e42780-f554-4797-a989-d526df665080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:07.067082 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37f154-9611-4454-9210-acacb0181c80 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:07.069542 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-5c37f154-9611-4454-9210-acacb0181c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:07.087963 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1398/2797] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:07.095664 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e42780-f554-4797-a989-d526df665080 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:07.098423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e5e42780-f554-4797-a989-d526df665080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:07.129699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:07.147166 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37f154-9611-4454-9210-acacb0181c80 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:07.152548 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105da999-ef87-45a6-b82f-326103cf7275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:07.153253 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105da999-ef87-45a6-b82f-326103cf7275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:07.153533 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105da999-ef87-45a6-b82f-326103cf7275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:07.153806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105da999-ef87-45a6-b82f-326103cf7275 admin admin] PATH_INFO: `/v3/projects/a120d2cba9884f148185606778440fff` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.154987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105da999-ef87-45a6-b82f-326103cf7275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a120d2cba9884f148185606778440fff)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:07.156937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105da999-ef87-45a6-b82f-326103cf7275 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:07.158195 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e42780-f554-4797-a989-d526df665080 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:07.162255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae48734-7037-4e68-89b2-012c76919085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:07.163415 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae48734-7037-4e68-89b2-012c76919085 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:07.163700 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae48734-7037-4e68-89b2-012c76919085 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:07.163986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae48734-7037-4e68-89b2-012c76919085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.172685 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae48734-7037-4e68-89b2-012c76919085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:07.174161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae48734-7037-4e68-89b2-012c76919085 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:07.183392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1399/2798] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:07.205749 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-105da999-ef87-45a6-b82f-326103cf7275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a120d2cba9884f148185606778440fff', 'request_id': 'req-105da999-ef87-45a6-b82f-326103cf7275'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:07.214625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-105da999-ef87-45a6-b82f-326103cf7275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a120d2cba9884f148185606778440fff', 'request_id': 'req-105da999-ef87-45a6-b82f-326103cf7275'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:07.220729 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-105da999-ef87-45a6-b82f-326103cf7275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a120d2cba9884f148185606778440fff', 'request_id': 'req-105da999-ef87-45a6-b82f-326103cf7275'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:07.234093 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-105da999-ef87-45a6-b82f-326103cf7275 admin admin] The token cache is being invalidate because project a120d2cba9884f148185606778440fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:41:07.234382 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-105da999-ef87-45a6-b82f-326103cf7275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:07.235022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-105da999-ef87-45a6-b82f-326103cf7275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:07.236745 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1400/2799] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:41:07 2022] DELETE /identity/v3/projects/a120d2cba9884f148185606778440fff => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:41:07.242144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777cd2f-30b5-45d8-ab24-10e27e4936a8 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:07.242432 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777cd2f-30b5-45d8-ab24-10e27e4936a8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:07.242701 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777cd2f-30b5-45d8-ab24-10e27e4936a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.262185 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3777cd2f-30b5-45d8-ab24-10e27e4936a8 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:07.263457 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3777cd2f-30b5-45d8-ab24-10e27e4936a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:07.319428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:07.332568 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1400/2800] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:07.398279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a19628-44a5-4ca1-a1e8-09ff388ef986 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:07.398597 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a19628-44a5-4ca1-a1e8-09ff388ef986 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:07.398876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a19628-44a5-4ca1-a1e8-09ff388ef986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.420696 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-09a19628-44a5-4ca1-a1e8-09ff388ef986 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:07.421928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-09a19628-44a5-4ca1-a1e8-09ff388ef986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:07.442625 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1401/2801] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:07.442740 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777cd2f-30b5-45d8-ab24-10e27e4936a8 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:07.446743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777cd2f-30b5-45d8-ab24-10e27e4936a8 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:07.452580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3845e9-231c-4f35-a7bb-21d7c85c4bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:07.453327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3845e9-231c-4f35-a7bb-21d7c85c4bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:07.453705 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3845e9-231c-4f35-a7bb-21d7c85c4bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:07.454078 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3845e9-231c-4f35-a7bb-21d7c85c4bd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.454835 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3845e9-231c-4f35-a7bb-21d7c85c4bd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:41:07.455360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3845e9-231c-4f35-a7bb-21d7c85c4bd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:07.457095 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3845e9-231c-4f35-a7bb-21d7c85c4bd5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:07.470328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1401/2802] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:41:07.474535 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a19628-44a5-4ca1-a1e8-09ff388ef986 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:07.485584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a19628-44a5-4ca1-a1e8-09ff388ef986 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:07.495661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76719e40-739f-44ff-893e-63b45247e4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:07.496285 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76719e40-739f-44ff-893e-63b45247e4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:07.496572 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76719e40-739f-44ff-893e-63b45247e4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:07.496887 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76719e40-739f-44ff-893e-63b45247e4a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.497521 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76719e40-739f-44ff-893e-63b45247e4a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:07.499575 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76719e40-739f-44ff-893e-63b45247e4a1 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:07.529072 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1402/2803] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:41:07 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:41:07.533304 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3845e9-231c-4f35-a7bb-21d7c85c4bd5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:07.545687 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3845e9-231c-4f35-a7bb-21d7c85c4bd5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:07.603357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad40e3-ffb6-435e-b322-0eb5362ad550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:07.603886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad40e3-ffb6-435e-b322-0eb5362ad550 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:07.604168 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad40e3-ffb6-435e-b322-0eb5362ad550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:07.604444 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad40e3-ffb6-435e-b322-0eb5362ad550 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.605099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad40e3-ffb6-435e-b322-0eb5362ad550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:07.607018 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad40e3-ffb6-435e-b322-0eb5362ad550 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:07.627792 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1402/2804] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:41:07 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:41:07.632240 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76719e40-739f-44ff-893e-63b45247e4a1 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:07.642692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76719e40-739f-44ff-893e-63b45247e4a1 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:07.647854 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e057f-b1f2-4914-9324-99981c26c1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:07.648458 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e057f-b1f2-4914-9324-99981c26c1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:07.648739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e057f-b1f2-4914-9324-99981c26c1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:07.649028 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e057f-b1f2-4914-9324-99981c26c1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.649669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e057f-b1f2-4914-9324-99981c26c1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:07.651640 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e057f-b1f2-4914-9324-99981c26c1c4 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:07.663125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1403/2805] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:41:07.667260 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad40e3-ffb6-435e-b322-0eb5362ad550 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:07.677294 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad40e3-ffb6-435e-b322-0eb5362ad550 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:07.681986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69e5da-eb42-4c80-a31d-f50b39a1242c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:07.682582 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69e5da-eb42-4c80-a31d-f50b39a1242c admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:07.682867 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69e5da-eb42-4c80-a31d-f50b39a1242c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:07.683144 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69e5da-eb42-4c80-a31d-f50b39a1242c admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.683782 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69e5da-eb42-4c80-a31d-f50b39a1242c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:07.685677 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69e5da-eb42-4c80-a31d-f50b39a1242c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:07.697368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1403/2806] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:41:07.701319 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e057f-b1f2-4914-9324-99981c26c1c4 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:07.711911 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e057f-b1f2-4914-9324-99981c26c1c4 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:07.716631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321baa3a-3a37-47ba-bea5-c79b77598991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:07.717309 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321baa3a-3a37-47ba-bea5-c79b77598991 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:07.717589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321baa3a-3a37-47ba-bea5-c79b77598991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:07.717863 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321baa3a-3a37-47ba-bea5-c79b77598991 admin admin] PATH_INFO: `/v3/projects/363667c09d4e481da5767f1043d57e76/users/886a581760e547b6a15856da84ef3859/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.733015 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321baa3a-3a37-47ba-bea5-c79b77598991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=363667c09d4e481da5767f1043d57e76, user_id=886a581760e547b6a15856da84ef3859, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:07.735658 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321baa3a-3a37-47ba-bea5-c79b77598991 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:07.748251 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1404/2807] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:41:07 2022] PUT /identity/v3/projects/363667c09d4e481da5767f1043d57e76/users/886a581760e547b6a15856da84ef3859/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:07.753477 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f81ec6-3b70-4489-8bb9-d30d9846f988 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:07.753768 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f81ec6-3b70-4489-8bb9-d30d9846f988 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:07.754048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f81ec6-3b70-4489-8bb9-d30d9846f988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.773947 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a6f81ec6-3b70-4489-8bb9-d30d9846f988 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:07.775153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a6f81ec6-3b70-4489-8bb9-d30d9846f988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:07.833416 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1404/2808] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:07.843492 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321baa3a-3a37-47ba-bea5-c79b77598991 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:07.845250 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-321baa3a-3a37-47ba-bea5-c79b77598991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:07.902730 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321baa3a-3a37-47ba-bea5-c79b77598991 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:07.906279 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1384c2-3b97-41bf-92e5-59ed1db4362b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:07.906876 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1384c2-3b97-41bf-92e5-59ed1db4362b service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:07.907152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1384c2-3b97-41bf-92e5-59ed1db4362b service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:07.907422 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1384c2-3b97-41bf-92e5-59ed1db4362b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:07.917093 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1384c2-3b97-41bf-92e5-59ed1db4362b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:07.918502 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1384c2-3b97-41bf-92e5-59ed1db4362b service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:07.928030 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1405/2809] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:10.829824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f81ec6-3b70-4489-8bb9-d30d9846f988 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:10.831342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a6f81ec6-3b70-4489-8bb9-d30d9846f988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:10.844383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f81ec6-3b70-4489-8bb9-d30d9846f988 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:10.849107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:10.850785 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:10.851142 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:10.851426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] PATH_INFO: `/v3/users/41db22367b0b454980e076378eb019be` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:10.852566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41db22367b0b454980e076378eb019be)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:10.854530 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:10.911589 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41db22367b0b454980e076378eb019be'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:10.913223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41db22367b0b454980e076378eb019be'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:10.914243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41db22367b0b454980e076378eb019be'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:10.924228 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:10.930635 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41db22367b0b454980e076378eb019be'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:10.940120 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41db22367b0b454980e076378eb019be'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:10.956786 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1405/2810] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:41:10 2022] DELETE /identity/v3/users/41db22367b0b454980e076378eb019be => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:10.963962 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15624cf-ddfc-4589-ad85-1117f0a43e08 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:10.965684 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15624cf-ddfc-4589-ad85-1117f0a43e08 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:10.966315 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15624cf-ddfc-4589-ad85-1117f0a43e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:10.994707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-b15624cf-ddfc-4589-ad85-1117f0a43e08 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:10.996010 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b15624cf-ddfc-4589-ad85-1117f0a43e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:11.016803 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:11.097637 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1406/2811] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:41:11.109668 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:11.111160 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:11.236253 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a394fb2-9a45-4ebc-8abc-cafd6dfb83a4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:11.242302 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f6bf5-f712-4ae8-9536-6e4165e17297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:11.243499 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f6bf5-f712-4ae8-9536-6e4165e17297 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:11.243944 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f6bf5-f712-4ae8-9536-6e4165e17297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:11.244369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f6bf5-f712-4ae8-9536-6e4165e17297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:11.276557 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f6bf5-f712-4ae8-9536-6e4165e17297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:11.278245 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f6bf5-f712-4ae8-9536-6e4165e17297 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:11.297769 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1406/2812] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:11.711986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15624cf-ddfc-4589-ad85-1117f0a43e08 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:11.713449 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-b15624cf-ddfc-4589-ad85-1117f0a43e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:11.751872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15624cf-ddfc-4589-ad85-1117f0a43e08 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:11.759376 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ca064-4657-4f87-ac87-3fe5c73195ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:11.760107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ca064-4657-4f87-ac87-3fe5c73195ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:11.760292 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ca064-4657-4f87-ac87-3fe5c73195ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:11.760560 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ca064-4657-4f87-ac87-3fe5c73195ed admin admin] PATH_INFO: `/v3/projects/f9697dfddc5f4b4fa66d39e2e87e1bdc` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:11.762088 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ca064-4657-4f87-ac87-3fe5c73195ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9697dfddc5f4b4fa66d39e2e87e1bdc)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:11.764233 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ca064-4657-4f87-ac87-3fe5c73195ed admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:11.823179 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-865ca064-4657-4f87-ac87-3fe5c73195ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9697dfddc5f4b4fa66d39e2e87e1bdc', 'request_id': 'req-865ca064-4657-4f87-ac87-3fe5c73195ed'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:11.829908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-865ca064-4657-4f87-ac87-3fe5c73195ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9697dfddc5f4b4fa66d39e2e87e1bdc', 'request_id': 'req-865ca064-4657-4f87-ac87-3fe5c73195ed'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:11.836439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-865ca064-4657-4f87-ac87-3fe5c73195ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9697dfddc5f4b4fa66d39e2e87e1bdc', 'request_id': 'req-865ca064-4657-4f87-ac87-3fe5c73195ed'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:11.851445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-865ca064-4657-4f87-ac87-3fe5c73195ed admin admin] The token cache is being invalidate because project f9697dfddc5f4b4fa66d39e2e87e1bdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:41:11.851739 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-865ca064-4657-4f87-ac87-3fe5c73195ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:11.852427 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-865ca064-4657-4f87-ac87-3fe5c73195ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:11.854450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1407/2813] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:41:11 2022] DELETE /identity/v3/projects/f9697dfddc5f4b4fa66d39e2e87e1bdc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:11.859757 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a5957-d9a1-4edd-9a5b-52bb03cbfc73 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:11.860270 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a5957-d9a1-4edd-9a5b-52bb03cbfc73 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:11.860368 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a5957-d9a1-4edd-9a5b-52bb03cbfc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:11.881682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-841a5957-d9a1-4edd-9a5b-52bb03cbfc73 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:11.882958 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-841a5957-d9a1-4edd-9a5b-52bb03cbfc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:11.959521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1407/2814] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:11.967180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e3775-303c-4193-b7e8-ecbe07440b7e None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:11.968044 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e3775-303c-4193-b7e8-ecbe07440b7e None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:11.972741 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e3775-303c-4193-b7e8-ecbe07440b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.018840 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-d42e3775-303c-4193-b7e8-ecbe07440b7e None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:12.020071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-d42e3775-303c-4193-b7e8-ecbe07440b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:12.031657 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1408/2815] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:12.033729 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a5957-d9a1-4edd-9a5b-52bb03cbfc73 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.052424 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a5957-d9a1-4edd-9a5b-52bb03cbfc73 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.059195 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5fc3c0-8b59-47ea-9da6-a416fb28e37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.060090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fc3c0-8b59-47ea-9da6-a416fb28e37d admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.060369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fc3c0-8b59-47ea-9da6-a416fb28e37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.060648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fc3c0-8b59-47ea-9da6-a416fb28e37d admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.061392 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5fc3c0-8b59-47ea-9da6-a416fb28e37d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:41:12.061712 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5fc3c0-8b59-47ea-9da6-a416fb28e37d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.063359 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5fc3c0-8b59-47ea-9da6-a416fb28e37d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.081900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1408/2816] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:41:12.086992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e3775-303c-4193-b7e8-ecbe07440b7e None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.101878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e3775-303c-4193-b7e8-ecbe07440b7e None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.109549 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d8055-9d51-4abb-b929-bd848c0d5d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.110528 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d8055-9d51-4abb-b929-bd848c0d5d6e admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.110964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d8055-9d51-4abb-b929-bd848c0d5d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.111391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d8055-9d51-4abb-b929-bd848c0d5d6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.112206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d8055-9d51-4abb-b929-bd848c0d5d6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.114405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d8055-9d51-4abb-b929-bd848c0d5d6e admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.160410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1409/2817] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:41:12 2022] POST /identity/v3/projects => generated 369 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:41:12.164598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5fc3c0-8b59-47ea-9da6-a416fb28e37d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.175647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5fc3c0-8b59-47ea-9da6-a416fb28e37d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.218135 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b3d15-4bbb-495c-b2c4-f78502c1bc07 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.219180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b3d15-4bbb-495c-b2c4-f78502c1bc07 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.219536 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b3d15-4bbb-495c-b2c4-f78502c1bc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.262423 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ea6b3d15-4bbb-495c-b2c4-f78502c1bc07 None None] MFA Rules not processed for user `886a581760e547b6a15856da84ef3859`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:12.263827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ea6b3d15-4bbb-495c-b2c4-f78502c1bc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:12.265041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38767bd0-196a-4e4c-8089-b0d2f8d0c967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.265940 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38767bd0-196a-4e4c-8089-b0d2f8d0c967 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.266710 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38767bd0-196a-4e4c-8089-b0d2f8d0c967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.267173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38767bd0-196a-4e4c-8089-b0d2f8d0c967 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.272930 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38767bd0-196a-4e4c-8089-b0d2f8d0c967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.275183 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38767bd0-196a-4e4c-8089-b0d2f8d0c967 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.286665 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:12.301138 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1409/2818] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:41:12 2022] POST /identity/v3/users => generated 285 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:41:12.302318 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38767bd0-196a-4e4c-8089-b0d2f8d0c967 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.306386 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-38767bd0-196a-4e4c-8089-b0d2f8d0c967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:12.321111 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38767bd0-196a-4e4c-8089-b0d2f8d0c967 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.326189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.326648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.326927 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.327199 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] PATH_INFO: `/v3/users/88a674908fda48b5b2ed89d596e55be7` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.328357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88a674908fda48b5b2ed89d596e55be7)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.330315 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.348926 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1410/2819] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:12.349817 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b3d15-4bbb-495c-b2c4-f78502c1bc07 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.355405 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:12.362534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b3d15-4bbb-495c-b2c4-f78502c1bc07 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.370512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dcdab-bd86-49cf-8e7e-7de6bd0137d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.371387 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dcdab-bd86-49cf-8e7e-7de6bd0137d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.371872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dcdab-bd86-49cf-8e7e-7de6bd0137d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.372172 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dcdab-bd86-49cf-8e7e-7de6bd0137d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.373158 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26dcdab-bd86-49cf-8e7e-7de6bd0137d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.375669 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26dcdab-bd86-49cf-8e7e-7de6bd0137d2 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.387599 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1411/2820] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:41:12.388510 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dcdab-bd86-49cf-8e7e-7de6bd0137d2 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.390339 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f26dcdab-bd86-49cf-8e7e-7de6bd0137d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:12.395210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88a674908fda48b5b2ed89d596e55be7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:12.395892 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88a674908fda48b5b2ed89d596e55be7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:12.396566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88a674908fda48b5b2ed89d596e55be7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:12.404366 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dcdab-bd86-49cf-8e7e-7de6bd0137d2 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.409359 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9c7cf-2eb7-43d7-bb00-5fbe9106b3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.409969 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9c7cf-2eb7-43d7-bb00-5fbe9106b3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.410248 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9c7cf-2eb7-43d7-bb00-5fbe9106b3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.410520 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9c7cf-2eb7-43d7-bb00-5fbe9106b3c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.410966 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88a674908fda48b5b2ed89d596e55be7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:12.411323 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9c7cf-2eb7-43d7-bb00-5fbe9106b3c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.413828 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9c7cf-2eb7-43d7-bb00-5fbe9106b3c0 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.421655 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88a674908fda48b5b2ed89d596e55be7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:12.438284 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1410/2821] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:41:12 2022] DELETE /identity/v3/users/88a674908fda48b5b2ed89d596e55be7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:12.438923 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.440580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:12.450460 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1412/2822] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:41:12 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:41:12.451904 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d28b5-0d59-4edc-b49c-0818d00ea477 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.452300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d28b5-0d59-4edc-b49c-0818d00ea477 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.452649 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d28b5-0d59-4edc-b49c-0818d00ea477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.472787 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-0c8d28b5-0d59-4edc-b49c-0818d00ea477 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:12.474124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0c8d28b5-0d59-4edc-b49c-0818d00ea477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:12.522666 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870d21e-6412-4e78-90b9-63e0393f9083 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.543291 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1413/2823] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:12.545648 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d28b5-0d59-4edc-b49c-0818d00ea477 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.547342 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-0c8d28b5-0d59-4edc-b49c-0818d00ea477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:12.562320 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d28b5-0d59-4edc-b49c-0818d00ea477 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.567253 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a13bc1-26a3-4484-aea8-a4431b9a6682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.567866 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a13bc1-26a3-4484-aea8-a4431b9a6682 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.568711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a13bc1-26a3-4484-aea8-a4431b9a6682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.569059 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a13bc1-26a3-4484-aea8-a4431b9a6682 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.569748 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a13bc1-26a3-4484-aea8-a4431b9a6682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.571820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a13bc1-26a3-4484-aea8-a4431b9a6682 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.596648 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e495088-9869-48aa-a3de-9a0e44cd3f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.597310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e495088-9869-48aa-a3de-9a0e44cd3f1e admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.597596 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e495088-9869-48aa-a3de-9a0e44cd3f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.597873 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e495088-9869-48aa-a3de-9a0e44cd3f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.598521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e495088-9869-48aa-a3de-9a0e44cd3f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.600983 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e495088-9869-48aa-a3de-9a0e44cd3f1e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.601512 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1414/2824] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:41:12 2022] POST /identity/v3/users => generated 292 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:41:12.602585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a13bc1-26a3-4484-aea8-a4431b9a6682 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.605173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-24a13bc1-26a3-4484-aea8-a4431b9a6682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:12.611613 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1411/2825] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:41:12.612469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e495088-9869-48aa-a3de-9a0e44cd3f1e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.623591 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e495088-9869-48aa-a3de-9a0e44cd3f1e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.628452 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af118362-587e-4b6d-9e90-e673769bae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.629115 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af118362-587e-4b6d-9e90-e673769bae3f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.629441 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af118362-587e-4b6d-9e90-e673769bae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.629680 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af118362-587e-4b6d-9e90-e673769bae3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.630320 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af118362-587e-4b6d-9e90-e673769bae3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.632410 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af118362-587e-4b6d-9e90-e673769bae3f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.637762 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:12.652714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1412/2826] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:41:12.652714 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af118362-587e-4b6d-9e90-e673769bae3f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.669255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af118362-587e-4b6d-9e90-e673769bae3f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.674566 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a9041-dacb-46a9-9cd0-f87435d2c11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.675296 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a9041-dacb-46a9-9cd0-f87435d2c11e admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.675443 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a13bc1-26a3-4484-aea8-a4431b9a6682 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.675539 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a9041-dacb-46a9-9cd0-f87435d2c11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.675811 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a9041-dacb-46a9-9cd0-f87435d2c11e admin admin] PATH_INFO: `/v3/projects/19c83ce3bc4b4b7cb21c8faac038acec/users/e3791a62e51247bc9e7a96acfbe309f9/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.680872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badac2ef-1032-49f8-a640-ba10afef7c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.681762 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badac2ef-1032-49f8-a640-ba10afef7c6a service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.681762 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badac2ef-1032-49f8-a640-ba10afef7c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.682138 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badac2ef-1032-49f8-a640-ba10afef7c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.695485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badac2ef-1032-49f8-a640-ba10afef7c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.697000 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badac2ef-1032-49f8-a640-ba10afef7c6a service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.705562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a9041-dacb-46a9-9cd0-f87435d2c11e admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c83ce3bc4b4b7cb21c8faac038acec, user_id=e3791a62e51247bc9e7a96acfbe309f9, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.708303 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1415/2827] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:12.709255 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a9041-dacb-46a9-9cd0-f87435d2c11e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.709941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badac2ef-1032-49f8-a640-ba10afef7c6a service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.723403 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badac2ef-1032-49f8-a640-ba10afef7c6a service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.730647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1413/2828] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:41:12 2022] PUT /identity/v3/projects/19c83ce3bc4b4b7cb21c8faac038acec/users/e3791a62e51247bc9e7a96acfbe309f9/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:12.792631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:12.813766 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f02954-99ae-4c98-8798-6fc2c3490bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.815208 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f02954-99ae-4c98-8798-6fc2c3490bec admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.815868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f02954-99ae-4c98-8798-6fc2c3490bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.816427 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f02954-99ae-4c98-8798-6fc2c3490bec admin admin] PATH_INFO: `/v3/projects/85e3f45084f140fd9cd6dbcbb1f0eaa2/users/6e778c57ba4949ec8d214aa138740b84/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.824152 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b627c-9f9f-446b-840c-b2ce9644ddb6 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.824620 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b627c-9f9f-446b-840c-b2ce9644ddb6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.825009 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b627c-9f9f-446b-840c-b2ce9644ddb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.849869 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f02954-99ae-4c98-8798-6fc2c3490bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=85e3f45084f140fd9cd6dbcbb1f0eaa2, user_id=6e778c57ba4949ec8d214aa138740b84, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.852706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f02954-99ae-4c98-8798-6fc2c3490bec admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.870576 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f13b627c-9f9f-446b-840c-b2ce9644ddb6 None None] MFA Rules not processed for user `e3791a62e51247bc9e7a96acfbe309f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:12.871747 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f13b627c-9f9f-446b-840c-b2ce9644ddb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:12.875162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1416/2829] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:41:12 2022] PUT /identity/v3/projects/85e3f45084f140fd9cd6dbcbb1f0eaa2/users/6e778c57ba4949ec8d214aa138740b84/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:12.879223 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f02954-99ae-4c98-8798-6fc2c3490bec admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.961025 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f02954-99ae-4c98-8798-6fc2c3490bec admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:12.969052 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4b84-2c6e-4eb7-9e75-61ec334a9657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:12.971278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1414/2830] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:41:12.971636 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4b84-2c6e-4eb7-9e75-61ec334a9657 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:12.971987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4b84-2c6e-4eb7-9e75-61ec334a9657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:12.972266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4b84-2c6e-4eb7-9e75-61ec334a9657 admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:12.972953 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd4b84-2c6e-4eb7-9e75-61ec334a9657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:12.975236 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd4b84-2c6e-4eb7-9e75-61ec334a9657 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:12.988950 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b627c-9f9f-446b-840c-b2ce9644ddb6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:12.993506 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f13b627c-9f9f-446b-840c-b2ce9644ddb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:12.995432 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1417/2831] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:41:12.996631 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4b84-2c6e-4eb7-9e75-61ec334a9657 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:13.017189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4b84-2c6e-4eb7-9e75-61ec334a9657 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:13.037878 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ad50a-ae59-4943-8597-35b149c20047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:13.042663 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ad50a-ae59-4943-8597-35b149c20047 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:13.044986 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ad50a-ae59-4943-8597-35b149c20047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:13.045302 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ad50a-ae59-4943-8597-35b149c20047 admin admin] PATH_INFO: `/v3/projects/85e3f45084f140fd9cd6dbcbb1f0eaa2/users/6e778c57ba4949ec8d214aa138740b84/roles/6ea33d7f5f9d47509b0153ec772527df` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:13.048056 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ad50a-ae59-4943-8597-35b149c20047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85e3f45084f140fd9cd6dbcbb1f0eaa2, user_id=6e778c57ba4949ec8d214aa138740b84, role_id=6ea33d7f5f9d47509b0153ec772527df)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:13.050862 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ad50a-ae59-4943-8597-35b149c20047 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:13.077206 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1418/2832] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:41:12 2022] PUT /identity/v3/projects/85e3f45084f140fd9cd6dbcbb1f0eaa2/users/6e778c57ba4949ec8d214aa138740b84/roles/6ea33d7f5f9d47509b0153ec772527df => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:13.079274 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ad50a-ae59-4943-8597-35b149c20047 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:13.084230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e40ad50a-ae59-4943-8597-35b149c20047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:13.096501 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b627c-9f9f-446b-840c-b2ce9644ddb6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:13.179838 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ad50a-ae59-4943-8597-35b149c20047 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:13.180226 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae5f74-da87-403e-a7f9-dcb549d8a634 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '748ad823b944441d907994cda6d8ac7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:13.181003 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae5f74-da87-403e-a7f9-dcb549d8a634 service glance] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:13.181288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae5f74-da87-403e-a7f9-dcb549d8a634 service glance] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:13.181563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae5f74-da87-403e-a7f9-dcb549d8a634 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:13.185972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:13.186868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:13.187231 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:13.187585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] PATH_INFO: `/v3/projects/095d7bf0a7524b5a8033663f1c3e7988` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:13.189067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=095d7bf0a7524b5a8033663f1c3e7988)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:13.191531 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:13.230935 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:13.272900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae5f74-da87-403e-a7f9-dcb549d8a634 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:13.274369 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae5f74-da87-403e-a7f9-dcb549d8a634 service glance] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:13.283428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '095d7bf0a7524b5a8033663f1c3e7988', 'request_id': 'req-c4b9538c-ab51-444b-a32a-5744035950f1'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:13.297435 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1415/2833] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:41:13.298740 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '095d7bf0a7524b5a8033663f1c3e7988', 'request_id': 'req-c4b9538c-ab51-444b-a32a-5744035950f1'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:13.308908 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '095d7bf0a7524b5a8033663f1c3e7988', 'request_id': 'req-c4b9538c-ab51-444b-a32a-5744035950f1'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:13.312007 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:13.335693 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] The token cache is being invalidate because project 095d7bf0a7524b5a8033663f1c3e7988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:41:13.336488 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:13.337753 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:13.344293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1419/2834] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:41:13 2022] DELETE /identity/v3/projects/095d7bf0a7524b5a8033663f1c3e7988 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:18.185366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922c378-3f23-4b3c-9ea0-bc7d4ac9d72b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:18.185894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922c378-3f23-4b3c-9ea0-bc7d4ac9d72b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:18.186002 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922c378-3f23-4b3c-9ea0-bc7d4ac9d72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:18.220224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2922c378-3f23-4b3c-9ea0-bc7d4ac9d72b None None] MFA Rules not processed for user `6e778c57ba4949ec8d214aa138740b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:18.221563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2922c378-3f23-4b3c-9ea0-bc7d4ac9d72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:18.316917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1416/2835] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:18.333245 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:18.334999 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:18.415706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9538c-ab51-444b-a32a-5744035950f1 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:18.421507 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2b92-eb28-4031-b37d-d9f028ea961b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:18.422189 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2b92-eb28-4031-b37d-d9f028ea961b service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:18.422479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2b92-eb28-4031-b37d-d9f028ea961b service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:18.422756 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2b92-eb28-4031-b37d-d9f028ea961b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:18.424684 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7bac2b92-eb28-4031-b37d-d9f028ea961b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:18.487506 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac2b92-eb28-4031-b37d-d9f028ea961b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:18.489170 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac2b92-eb28-4031-b37d-d9f028ea961b service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:18.500970 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1420/2836] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:20.619638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eefbff-1c8a-4fe8-aba7-a87fa2f95808 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:20.620150 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eefbff-1c8a-4fe8-aba7-a87fa2f95808 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:20.620268 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eefbff-1c8a-4fe8-aba7-a87fa2f95808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:20.644874 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-32eefbff-1c8a-4fe8-aba7-a87fa2f95808 None None] MFA Rules not processed for user `886a581760e547b6a15856da84ef3859`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:20.647403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-32eefbff-1c8a-4fe8-aba7-a87fa2f95808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:20.656321 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1417/2837] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:28.495493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=58 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:30.635448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=48 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:41.439301 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2b92-eb28-4031-b37d-d9f028ea961b service nova] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:41.440742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7bac2b92-eb28-4031-b37d-d9f028ea961b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:41.507789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2b92-eb28-4031-b37d-d9f028ea961b service nova] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:41.512960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:41.513530 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:41.513837 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:41.514109 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] PATH_INFO: `/v3/users/e3791a62e51247bc9e7a96acfbe309f9` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:41.515228 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3791a62e51247bc9e7a96acfbe309f9)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:41.517159 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:41.564820 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3791a62e51247bc9e7a96acfbe309f9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:41.565479 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3791a62e51247bc9e7a96acfbe309f9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:41.566162 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3791a62e51247bc9e7a96acfbe309f9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:41.573644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3791a62e51247bc9e7a96acfbe309f9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:41.586355 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3791a62e51247bc9e7a96acfbe309f9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:41.600407 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1421/2838] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:41:41 2022] DELETE /identity/v3/users/e3791a62e51247bc9e7a96acfbe309f9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:41.604968 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b7f9f-a5fe-4239-a277-0c98eb47c19f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:41.605448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b7f9f-a5fe-4239-a277-0c98eb47c19f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:41.605630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b7f9f-a5fe-4239-a277-0c98eb47c19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:41.624521 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ef7b7f9f-a5fe-4239-a277-0c98eb47c19f None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:41.625824 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ef7b7f9f-a5fe-4239-a277-0c98eb47c19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:41.690363 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1418/2839] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:41.700418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:41.701907 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:41.760337 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229f740-a050-4b90-bcab-6c0085a6f213 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:41.763972 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651c707-a088-4dc2-a6f0-798df5c84be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:41.764517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651c707-a088-4dc2-a6f0-798df5c84be5 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:41.764850 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651c707-a088-4dc2-a6f0-798df5c84be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:41.765112 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651c707-a088-4dc2-a6f0-798df5c84be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:41.775133 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651c707-a088-4dc2-a6f0-798df5c84be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:41.776550 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651c707-a088-4dc2-a6f0-798df5c84be5 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:41.785993 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1422/2840] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:42.106610 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b7f9f-a5fe-4239-a277-0c98eb47c19f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:42.108027 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ef7b7f9f-a5fe-4239-a277-0c98eb47c19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:42.123357 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b7f9f-a5fe-4239-a277-0c98eb47c19f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:42.129565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9acbf-1721-4058-bc48-9d323ce2a114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:42.130207 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9acbf-1721-4058-bc48-9d323ce2a114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:42.131212 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9acbf-1721-4058-bc48-9d323ce2a114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:42.131831 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9acbf-1721-4058-bc48-9d323ce2a114 admin admin] PATH_INFO: `/v3/projects/19c83ce3bc4b4b7cb21c8faac038acec` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:42.133193 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9acbf-1721-4058-bc48-9d323ce2a114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19c83ce3bc4b4b7cb21c8faac038acec)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:42.135324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9acbf-1721-4058-bc48-9d323ce2a114 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:42.187082 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-8fe9acbf-1721-4058-bc48-9d323ce2a114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19c83ce3bc4b4b7cb21c8faac038acec', 'request_id': 'req-8fe9acbf-1721-4058-bc48-9d323ce2a114'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:42.193746 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-8fe9acbf-1721-4058-bc48-9d323ce2a114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19c83ce3bc4b4b7cb21c8faac038acec', 'request_id': 'req-8fe9acbf-1721-4058-bc48-9d323ce2a114'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:42.200822 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-8fe9acbf-1721-4058-bc48-9d323ce2a114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19c83ce3bc4b4b7cb21c8faac038acec', 'request_id': 'req-8fe9acbf-1721-4058-bc48-9d323ce2a114'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:42.215393 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-8fe9acbf-1721-4058-bc48-9d323ce2a114 admin admin] The token cache is being invalidate because project 19c83ce3bc4b4b7cb21c8faac038acec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:41:42.215704 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-8fe9acbf-1721-4058-bc48-9d323ce2a114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:42.216322 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-8fe9acbf-1721-4058-bc48-9d323ce2a114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:41:42.218067 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1419/2841] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:41:42 2022] DELETE /identity/v3/projects/19c83ce3bc4b4b7cb21c8faac038acec => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:42.223429 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fe45a-2d34-4f76-bcf6-7ebf04b20327 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:42.223707 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fe45a-2d34-4f76-bcf6-7ebf04b20327 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:42.223977 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fe45a-2d34-4f76-bcf6-7ebf04b20327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:42.245711 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-7b6fe45a-2d34-4f76-bcf6-7ebf04b20327 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:42.246880 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7b6fe45a-2d34-4f76-bcf6-7ebf04b20327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:42.265438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:42.311947 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1423/2842] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:42.319911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df660e28-e892-4d91-b7a3-05f583bf746b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:42.320383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df660e28-e892-4d91-b7a3-05f583bf746b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:42.320885 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df660e28-e892-4d91-b7a3-05f583bf746b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:42.342671 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-df660e28-e892-4d91-b7a3-05f583bf746b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:42.344076 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-df660e28-e892-4d91-b7a3-05f583bf746b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:42.352937 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1420/2843] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:41:42.357360 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fe45a-2d34-4f76-bcf6-7ebf04b20327 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:42.367324 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fe45a-2d34-4f76-bcf6-7ebf04b20327 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:42.371886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52ffe3-6572-4ed4-8ab9-1de330087ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:42.372482 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52ffe3-6572-4ed4-8ab9-1de330087ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:42.372778 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52ffe3-6572-4ed4-8ab9-1de330087ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:42.373058 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52ffe3-6572-4ed4-8ab9-1de330087ec6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:42.373699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52ffe3-6572-4ed4-8ab9-1de330087ec6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81550) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:41:42.373998 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52ffe3-6572-4ed4-8ab9-1de330087ec6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:42.375589 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52ffe3-6572-4ed4-8ab9-1de330087ec6 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:42.388336 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1424/2844] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:41:42.394561 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df660e28-e892-4d91-b7a3-05f583bf746b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:42.404926 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df660e28-e892-4d91-b7a3-05f583bf746b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:42.410449 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f231b-3df2-4108-b6ca-7c50ca1f8cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:42.411074 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f231b-3df2-4108-b6ca-7c50ca1f8cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:42.411352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f231b-3df2-4108-b6ca-7c50ca1f8cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:42.411630 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f231b-3df2-4108-b6ca-7c50ca1f8cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:42.412259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f231b-3df2-4108-b6ca-7c50ca1f8cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:42.414210 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f231b-3df2-4108-b6ca-7c50ca1f8cf7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:42.453723 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1421/2845] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:41:42 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 16:41:42.458927 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52ffe3-6572-4ed4-8ab9-1de330087ec6 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:42.469143 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52ffe3-6572-4ed4-8ab9-1de330087ec6 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:42.528613 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197e21e-173b-4308-ab6e-e48594a3ed71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:42.529252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197e21e-173b-4308-ab6e-e48594a3ed71 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:42.529635 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197e21e-173b-4308-ab6e-e48594a3ed71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:42.529905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197e21e-173b-4308-ab6e-e48594a3ed71 admin admin] PATH_INFO: `/v3/users` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:42.530527 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197e21e-173b-4308-ab6e-e48594a3ed71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:42.532440 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197e21e-173b-4308-ab6e-e48594a3ed71 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:42.554993 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1425/2846] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:41:42 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:41:42.559389 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f231b-3df2-4108-b6ca-7c50ca1f8cf7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:42.570278 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f231b-3df2-4108-b6ca-7c50ca1f8cf7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:42.575214 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ab417-9a91-4a08-bf95-082954116052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:42.575851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ab417-9a91-4a08-bf95-082954116052 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:42.576106 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ab417-9a91-4a08-bf95-082954116052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:42.576419 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ab417-9a91-4a08-bf95-082954116052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:42.577146 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ab417-9a91-4a08-bf95-082954116052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:42.578896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ab417-9a91-4a08-bf95-082954116052 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:42.589329 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1422/2847] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:41:42.594029 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197e21e-173b-4308-ab6e-e48594a3ed71 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:42.605439 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197e21e-173b-4308-ab6e-e48594a3ed71 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:42.610485 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddef9ba-1b19-4a44-ae84-4503d711057b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:42.611788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddef9ba-1b19-4a44-ae84-4503d711057b admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:42.612074 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddef9ba-1b19-4a44-ae84-4503d711057b admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:42.612691 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddef9ba-1b19-4a44-ae84-4503d711057b admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:42.613942 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddef9ba-1b19-4a44-ae84-4503d711057b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:42.615546 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddef9ba-1b19-4a44-ae84-4503d711057b admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:42.628116 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1426/2848] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 16:41:42.632598 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ab417-9a91-4a08-bf95-082954116052 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:42.643223 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ab417-9a91-4a08-bf95-082954116052 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:42.648167 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ea7bb-611c-4196-8e30-a354ae860ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:42.648863 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ea7bb-611c-4196-8e30-a354ae860ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:42.649148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ea7bb-611c-4196-8e30-a354ae860ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:42.649423 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ea7bb-611c-4196-8e30-a354ae860ec7 admin admin] PATH_INFO: `/v3/projects/e552ac8505064742b40ef123a178facd/users/87af5fad51cf4e77a621ae410f77a0b5/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:42.665461 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ea7bb-611c-4196-8e30-a354ae860ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e552ac8505064742b40ef123a178facd, user_id=87af5fad51cf4e77a621ae410f77a0b5, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:42.668107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ea7bb-611c-4196-8e30-a354ae860ec7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:42.681647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1423/2849] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:41:42 2022] PUT /identity/v3/projects/e552ac8505064742b40ef123a178facd/users/87af5fad51cf4e77a621ae410f77a0b5/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:41:42.692404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79dfbf1-fffe-4bdb-ab85-40ea161c5257 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:42.692695 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79dfbf1-fffe-4bdb-ab85-40ea161c5257 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:42.693003 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79dfbf1-fffe-4bdb-ab85-40ea161c5257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:42.719567 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-e79dfbf1-fffe-4bdb-ab85-40ea161c5257 None None] MFA Rules not processed for user `87af5fad51cf4e77a621ae410f77a0b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:41:42.720800 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e79dfbf1-fffe-4bdb-ab85-40ea161c5257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:42.786290 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1427/2850] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:41:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:41:42.797360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ea7bb-611c-4196-8e30-a354ae860ec7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:42.800371 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-002ea7bb-611c-4196-8e30-a354ae860ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:42.814563 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:42.860327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ea7bb-611c-4196-8e30-a354ae860ec7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:42.864041 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c320d18-62ff-4433-b2bf-e8d242a520fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:42.864650 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c320d18-62ff-4433-b2bf-e8d242a520fe service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:42.864953 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c320d18-62ff-4433-b2bf-e8d242a520fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:42.865224 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c320d18-62ff-4433-b2bf-e8d242a520fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:42.874581 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c320d18-62ff-4433-b2bf-e8d242a520fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:42.876023 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c320d18-62ff-4433-b2bf-e8d242a520fe service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:42.885140 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1424/2851] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:49.067450 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79dfbf1-fffe-4bdb-ab85-40ea161c5257 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:49.068961 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-e79dfbf1-fffe-4bdb-ab85-40ea161c5257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:49.088650 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:49.126941 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79dfbf1-fffe-4bdb-ab85-40ea161c5257 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:49.131050 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fe3fc-6c63-42d2-8b6e-e5696030af87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:49.131643 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fe3fc-6c63-42d2-8b6e-e5696030af87 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:49.131918 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fe3fc-6c63-42d2-8b6e-e5696030af87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:49.132190 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fe3fc-6c63-42d2-8b6e-e5696030af87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:49.133917 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-975fe3fc-6c63-42d2-8b6e-e5696030af87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:49.203118 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fe3fc-6c63-42d2-8b6e-e5696030af87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:49.204585 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fe3fc-6c63-42d2-8b6e-e5696030af87 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:49.216910 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1428/2852] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:52.881065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=21 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:53.009941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c320d18-62ff-4433-b2bf-e8d242a520fe service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:53.011471 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6c320d18-62ff-4433-b2bf-e8d242a520fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:53.023571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c320d18-62ff-4433-b2bf-e8d242a520fe service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:53.027219 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7934249-c6ef-450f-b6a7-8874a26aab72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:53.027812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934249-c6ef-450f-b6a7-8874a26aab72 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:53.028092 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934249-c6ef-450f-b6a7-8874a26aab72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:53.028364 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934249-c6ef-450f-b6a7-8874a26aab72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:53.030071 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d7934249-c6ef-450f-b6a7-8874a26aab72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:53.044090 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7934249-c6ef-450f-b6a7-8874a26aab72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:53.045562 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7934249-c6ef-450f-b6a7-8874a26aab72 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:53.057804 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1425/2853] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:41:59.211391 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=46 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:41:59.249806 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fe3fc-6c63-42d2-8b6e-e5696030af87 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:41:59.259081 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fe3fc-6c63-42d2-8b6e-e5696030af87 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:41:59.262825 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff0765-bd3b-4fc3-96bf-a7a122293d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:41:59.263428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff0765-bd3b-4fc3-96bf-a7a122293d4d service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:41:59.263706 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff0765-bd3b-4fc3-96bf-a7a122293d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:41:59.263980 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff0765-bd3b-4fc3-96bf-a7a122293d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:41:59.265874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-f9ff0765-bd3b-4fc3-96bf-a7a122293d4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:41:59.333616 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff0765-bd3b-4fc3-96bf-a7a122293d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:41:59.335061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff0765-bd3b-4fc3-96bf-a7a122293d4d service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:41:59.344923 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1429/2854] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:41:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:42:03.052489 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=6 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:42:09.340561 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=30 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:42:17.980622 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7934249-c6ef-450f-b6a7-8874a26aab72 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:18.042645 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7934249-c6ef-450f-b6a7-8874a26aab72 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:18.047179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:18.047800 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:18.048079 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:18.048352 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] PATH_INFO: `/v3/users/87af5fad51cf4e77a621ae410f77a0b5` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:18.049508 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87af5fad51cf4e77a621ae410f77a0b5)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:18.051437 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:18.096567 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87af5fad51cf4e77a621ae410f77a0b5'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:18.097243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87af5fad51cf4e77a621ae410f77a0b5'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:18.097838 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87af5fad51cf4e77a621ae410f77a0b5'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:18.109344 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87af5fad51cf4e77a621ae410f77a0b5'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:18.116093 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87af5fad51cf4e77a621ae410f77a0b5'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:18.129060 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1426/2855] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:42:17 2022] DELETE /identity/v3/users/87af5fad51cf4e77a621ae410f77a0b5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:42:18.133627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe38605-0931-491c-8b46-8d19eff6701b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:18.134101 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe38605-0931-491c-8b46-8d19eff6701b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:18.134201 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe38605-0931-491c-8b46-8d19eff6701b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:18.153573 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-cfe38605-0931-491c-8b46-8d19eff6701b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:42:18.154768 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-cfe38605-0931-491c-8b46-8d19eff6701b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:18.212743 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1430/2856] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:42:18.223707 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:18.225310 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:18.282663 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b352bb-4a2a-4a6f-af05-abe0bd81fcc9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:18.286292 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc07251-ca93-4871-bab7-eb8e3c5e3f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:18.286896 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc07251-ca93-4871-bab7-eb8e3c5e3f81 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:18.287173 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc07251-ca93-4871-bab7-eb8e3c5e3f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:18.287448 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc07251-ca93-4871-bab7-eb8e3c5e3f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:18.296413 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc07251-ca93-4871-bab7-eb8e3c5e3f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:18.297887 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc07251-ca93-4871-bab7-eb8e3c5e3f81 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:18.308179 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1427/2857] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:42:18.575385 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe38605-0931-491c-8b46-8d19eff6701b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:18.576981 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-cfe38605-0931-491c-8b46-8d19eff6701b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:18.593263 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe38605-0931-491c-8b46-8d19eff6701b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:18.598603 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:18.599252 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:18.599529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:18.599891 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f admin admin] PATH_INFO: `/v3/projects/e552ac8505064742b40ef123a178facd` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:18.601042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e552ac8505064742b40ef123a178facd)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:18.602992 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:18.650266 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e552ac8505064742b40ef123a178facd', 'request_id': 'req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:18.656642 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e552ac8505064742b40ef123a178facd', 'request_id': 'req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:18.662945 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e552ac8505064742b40ef123a178facd', 'request_id': 'req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:18.675950 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f admin admin] The token cache is being invalidate because project e552ac8505064742b40ef123a178facd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:42:18.676230 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:18.676872 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-e11328ff-e6b1-4364-8f6a-ca83ff7ab98f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:18.678365 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1431/2858] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:42:18 2022] DELETE /identity/v3/projects/e552ac8505064742b40ef123a178facd => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:42:18.683502 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997820c-3d1e-4188-a9a1-14080f818bd3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:18.683880 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997820c-3d1e-4188-a9a1-14080f818bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:18.684148 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997820c-3d1e-4188-a9a1-14080f818bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:18.703319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8997820c-3d1e-4188-a9a1-14080f818bd3 None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:42:18.704522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8997820c-3d1e-4188-a9a1-14080f818bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:18.721524 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:42:18.764593 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1428/2859] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:42:18.772801 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c476ac1-74cc-4da1-bc60-26392b9e868c None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:18.773089 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c476ac1-74cc-4da1-bc60-26392b9e868c None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:18.773361 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c476ac1-74cc-4da1-bc60-26392b9e868c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:18.793156 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-7c476ac1-74cc-4da1-bc60-26392b9e868c None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:42:18.794350 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-7c476ac1-74cc-4da1-bc60-26392b9e868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:18.802235 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1432/2860] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:42:18.806438 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997820c-3d1e-4188-a9a1-14080f818bd3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:18.816231 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997820c-3d1e-4188-a9a1-14080f818bd3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:18.820744 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac39d4-882d-4c73-bc8c-e0a16f7b0358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:18.821366 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac39d4-882d-4c73-bc8c-e0a16f7b0358 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:18.821641 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac39d4-882d-4c73-bc8c-e0a16f7b0358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:18.821911 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac39d4-882d-4c73-bc8c-e0a16f7b0358 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:18.822544 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac39d4-882d-4c73-bc8c-e0a16f7b0358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 16:42:18.822848 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac39d4-882d-4c73-bc8c-e0a16f7b0358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:18.824451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac39d4-882d-4c73-bc8c-e0a16f7b0358 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:18.836461 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1429/2861] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Aug 19 16:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 16:42:18.840529 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c476ac1-74cc-4da1-bc60-26392b9e868c None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:18.850416 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c476ac1-74cc-4da1-bc60-26392b9e868c None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:18.855104 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd35d9e-8479-447f-ac9d-6823904cf492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:18.855699 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd35d9e-8479-447f-ac9d-6823904cf492 admin admin] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:18.855977 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd35d9e-8479-447f-ac9d-6823904cf492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:18.856247 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd35d9e-8479-447f-ac9d-6823904cf492 admin admin] PATH_INFO: `/v3/projects` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:18.856899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd35d9e-8479-447f-ac9d-6823904cf492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:18.858805 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd35d9e-8479-447f-ac9d-6823904cf492 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:18.885789 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1433/2862] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Aug 19 16:42:18 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:42:18.889986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac39d4-882d-4c73-bc8c-e0a16f7b0358 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:18.900739 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac39d4-882d-4c73-bc8c-e0a16f7b0358 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:18.957969 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16458f38-bff9-4309-b67b-2aa5994fb919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:18.958571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16458f38-bff9-4309-b67b-2aa5994fb919 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:18.958849 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16458f38-bff9-4309-b67b-2aa5994fb919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:18.959121 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16458f38-bff9-4309-b67b-2aa5994fb919 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:18.959738 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16458f38-bff9-4309-b67b-2aa5994fb919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:18.961688 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16458f38-bff9-4309-b67b-2aa5994fb919 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:18.983403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1430/2863] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Aug 19 16:42:18 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 16:42:18.988269 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd35d9e-8479-447f-ac9d-6823904cf492 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:18.998607 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd35d9e-8479-447f-ac9d-6823904cf492 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:19.004173 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc536ff-767e-484e-91c6-726d6996704d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:19.004793 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc536ff-767e-484e-91c6-726d6996704d admin admin] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:19.005075 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc536ff-767e-484e-91c6-726d6996704d admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:19.005351 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc536ff-767e-484e-91c6-726d6996704d admin admin] PATH_INFO: `/v3/roles` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:19.005987 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc536ff-767e-484e-91c6-726d6996704d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:19.007580 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc536ff-767e-484e-91c6-726d6996704d admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:19.017441 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1434/2864] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:42:19.021287 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16458f38-bff9-4309-b67b-2aa5994fb919 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:19.031565 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16458f38-bff9-4309-b67b-2aa5994fb919 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:19.036477 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cbf26-f88a-43d1-b861-99d628fc84fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:19.037103 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cbf26-f88a-43d1-b861-99d628fc84fc admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:19.037383 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cbf26-f88a-43d1-b861-99d628fc84fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:19.037653 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cbf26-f88a-43d1-b861-99d628fc84fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:19.038280 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cbf26-f88a-43d1-b861-99d628fc84fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:19.039868 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cbf26-f88a-43d1-b861-99d628fc84fc admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:19.051251 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1431/2865] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Aug 19 16:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 16:42:19.055362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc536ff-767e-484e-91c6-726d6996704d admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:19.066261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc536ff-767e-484e-91c6-726d6996704d admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:19.070905 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7e136-d20f-4f83-a7d3-78ce1f54de82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:19.071553 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7e136-d20f-4f83-a7d3-78ce1f54de82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:19.071833 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7e136-d20f-4f83-a7d3-78ce1f54de82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:19.072107 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7e136-d20f-4f83-a7d3-78ce1f54de82 admin admin] PATH_INFO: `/v3/projects/aca4caddd5c640da85cdd583ba0e2230/users/d8735dccaf424439a24683f3533489dd/roles/ac5a835695db44918d768db1bbb97589` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:19.086425 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7e136-d20f-4f83-a7d3-78ce1f54de82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca4caddd5c640da85cdd583ba0e2230, user_id=d8735dccaf424439a24683f3533489dd, role_id=ac5a835695db44918d768db1bbb97589)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:19.089073 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7e136-d20f-4f83-a7d3-78ce1f54de82 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:19.101967 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1435/2866] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Aug 19 16:42:19 2022] PUT /identity/v3/projects/aca4caddd5c640da85cdd583ba0e2230/users/d8735dccaf424439a24683f3533489dd/roles/ac5a835695db44918d768db1bbb97589 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:42:19.114165 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100f103-c8ae-4c46-8879-c0859f7ac9c9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:19.114465 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100f103-c8ae-4c46-8879-c0859f7ac9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:19.114735 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100f103-c8ae-4c46-8879-c0859f7ac9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:19.142720 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8100f103-c8ae-4c46-8879-c0859f7ac9c9 None None] MFA Rules not processed for user `d8735dccaf424439a24683f3533489dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:42:19.144189 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8100f103-c8ae-4c46-8879-c0859f7ac9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:19.207288 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1432/2867] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:42:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 16:42:19.217874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7e136-d20f-4f83-a7d3-78ce1f54de82 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:19.219560 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-1ce7e136-d20f-4f83-a7d3-78ce1f54de82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:19.242140 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:42:19.278488 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7e136-d20f-4f83-a7d3-78ce1f54de82 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:19.282062 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1b65b-63cd-4404-a3ed-cab9d575a7af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:19.282659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1b65b-63cd-4404-a3ed-cab9d575a7af service cinder] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:19.282934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1b65b-63cd-4404-a3ed-cab9d575a7af service cinder] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:19.283202 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1b65b-63cd-4404-a3ed-cab9d575a7af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:19.292211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1b65b-63cd-4404-a3ed-cab9d575a7af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:19.293656 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1b65b-63cd-4404-a3ed-cab9d575a7af service cinder] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:19.303031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1436/2868] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:42:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:42:29.048257 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100f103-c8ae-4c46-8879-c0859f7ac9c9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:29.068188 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:42:29.112715 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100f103-c8ae-4c46-8879-c0859f7ac9c9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:29.117812 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:29.118451 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:29.118727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:29.118993 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] PATH_INFO: `/v3/users/d8735dccaf424439a24683f3533489dd` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:29.120096 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8735dccaf424439a24683f3533489dd)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:29.122042 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:29.167907 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8735dccaf424439a24683f3533489dd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:29.168551 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8735dccaf424439a24683f3533489dd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:29.169174 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8735dccaf424439a24683f3533489dd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:29.180709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8735dccaf424439a24683f3533489dd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:29.188403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8735dccaf424439a24683f3533489dd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:29.201007 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1433/2869] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:42:29 2022] DELETE /identity/v3/users/d8735dccaf424439a24683f3533489dd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:42:29.205454 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc67de-f5d1-4390-b560-6c0ba0faa64b None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:29.205944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc67de-f5d1-4390-b560-6c0ba0faa64b None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:29.206061 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc67de-f5d1-4390-b560-6c0ba0faa64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:29.225661 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-81fc67de-f5d1-4390-b560-6c0ba0faa64b None None] MFA Rules not processed for user `1decd3055af14f42987a890baebb81a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:42:29.227476 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-81fc67de-f5d1-4390-b560-6c0ba0faa64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:29.284807 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1437/2870] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:42:29.295230 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:29.296631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:29.354342 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ee937-be5a-4309-8740-86ac9c6372c5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:29.357906 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f69d9-2f6e-4e2c-b3ab-d77fd02954ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:29.358504 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f69d9-2f6e-4e2c-b3ab-d77fd02954ef service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:29.358779 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f69d9-2f6e-4e2c-b3ab-d77fd02954ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:29.359049 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f69d9-2f6e-4e2c-b3ab-d77fd02954ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:29.368327 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f69d9-2f6e-4e2c-b3ab-d77fd02954ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:29.369781 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f69d9-2f6e-4e2c-b3ab-d77fd02954ef service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:29.379116 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1434/2871] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:42:29.642516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc67de-f5d1-4390-b560-6c0ba0faa64b None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:29.644175 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-81fc67de-f5d1-4390-b560-6c0ba0faa64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:29.659122 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc67de-f5d1-4390-b560-6c0ba0faa64b None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:29.663896 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe02542-4e45-45f3-9cc9-0e36e3443256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:29.665437 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe02542-4e45-45f3-9cc9-0e36e3443256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:29.665864 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe02542-4e45-45f3-9cc9-0e36e3443256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:29.666145 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe02542-4e45-45f3-9cc9-0e36e3443256 admin admin] PATH_INFO: `/v3/projects/aca4caddd5c640da85cdd583ba0e2230` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:29.667294 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe02542-4e45-45f3-9cc9-0e36e3443256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca4caddd5c640da85cdd583ba0e2230)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:29.669242 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe02542-4e45-45f3-9cc9-0e36e3443256 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:29.717428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-3fe02542-4e45-45f3-9cc9-0e36e3443256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca4caddd5c640da85cdd583ba0e2230', 'request_id': 'req-3fe02542-4e45-45f3-9cc9-0e36e3443256'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:29.723383 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-3fe02542-4e45-45f3-9cc9-0e36e3443256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca4caddd5c640da85cdd583ba0e2230', 'request_id': 'req-3fe02542-4e45-45f3-9cc9-0e36e3443256'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:29.728936 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-3fe02542-4e45-45f3-9cc9-0e36e3443256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca4caddd5c640da85cdd583ba0e2230', 'request_id': 'req-3fe02542-4e45-45f3-9cc9-0e36e3443256'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:29.740960 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-3fe02542-4e45-45f3-9cc9-0e36e3443256 admin admin] The token cache is being invalidate because project aca4caddd5c640da85cdd583ba0e2230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81550) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:42:29.741249 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-3fe02542-4e45-45f3-9cc9-0e36e3443256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:29.741855 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-3fe02542-4e45-45f3-9cc9-0e36e3443256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:42:29.743411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1438/2872] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:42:29 2022] DELETE /identity/v3/projects/aca4caddd5c640da85cdd583ba0e2230 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:42:35.397839 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f69d9-2f6e-4e2c-b3ab-d77fd02954ef service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:35.399227 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0e6f69d9-2f6e-4e2c-b3ab-d77fd02954ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:35.443900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:42:35.459336 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f69d9-2f6e-4e2c-b3ab-d77fd02954ef service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:35.507283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df964888-90c9-4903-8d1f-dcdae8383f56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eaa3980ad8840879e4e426b00e3c56f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:35.507886 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df964888-90c9-4903-8d1f-dcdae8383f56 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:35.508165 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df964888-90c9-4903-8d1f-dcdae8383f56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:35.508434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df964888-90c9-4903-8d1f-dcdae8383f56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:35.510741 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-df964888-90c9-4903-8d1f-dcdae8383f56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:35.569203 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df964888-90c9-4903-8d1f-dcdae8383f56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:35.570660 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df964888-90c9-4903-8d1f-dcdae8383f56 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:35.580267 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1435/2873] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:42:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 16:42:39.737924 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:42:44.496516 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5d2a1-b08c-4c1a-8138-e945cd35cd75 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:44.497777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5d2a1-b08c-4c1a-8138-e945cd35cd75 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:44.497777 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5d2a1-b08c-4c1a-8138-e945cd35cd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:44.516237 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-ebc5d2a1-b08c-4c1a-8138-e945cd35cd75 None None] MFA Rules not processed for user `f989b6fded53458b85b39789a32a39e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:42:44.517462 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ebc5d2a1-b08c-4c1a-8138-e945cd35cd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:44.524152 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1439/2874] 10.222.0.55 () {58 vars in 958 bytes} [Fri Aug 19 16:42:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:42:45.576096 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=50 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:42:46.277422 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df964888-90c9-4903-8d1f-dcdae8383f56 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:46.279679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-df964888-90c9-4903-8d1f-dcdae8383f56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:46.351900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df964888-90c9-4903-8d1f-dcdae8383f56 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:46.357164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69192234-00f9-4d19-b9a8-0fb69fb25963 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:46.357770 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69192234-00f9-4d19-b9a8-0fb69fb25963 service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:46.358048 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69192234-00f9-4d19-b9a8-0fb69fb25963 service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:46.358319 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69192234-00f9-4d19-b9a8-0fb69fb25963 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:46.360020 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-69192234-00f9-4d19-b9a8-0fb69fb25963 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:46.371202 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69192234-00f9-4d19-b9a8-0fb69fb25963 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:46.373684 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69192234-00f9-4d19-b9a8-0fb69fb25963 service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:46.384206 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1436/2875] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:42:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:42:48.235737 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5d2a1-b08c-4c1a-8138-e945cd35cd75 None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:48.237342 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ebc5d2a1-b08c-4c1a-8138-e945cd35cd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:48.310469 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5d2a1-b08c-4c1a-8138-e945cd35cd75 None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:48.314328 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5618a3-b6db-49d4-957d-17f8961e2ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:48.314928 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5618a3-b6db-49d4-957d-17f8961e2ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:48.315203 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5618a3-b6db-49d4-957d-17f8961e2ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:48.315473 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5618a3-b6db-49d4-957d-17f8961e2ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:48.317222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-bb5618a3-b6db-49d4-957d-17f8961e2ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:48.328574 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5618a3-b6db-49d4-957d-17f8961e2ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:48.330034 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5618a3-b6db-49d4-957d-17f8961e2ef5 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:48.338502 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1440/2876] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:42:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:42:48.917631 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69192234-00f9-4d19-b9a8-0fb69fb25963 service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:48.919065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-69192234-00f9-4d19-b9a8-0fb69fb25963 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:48.997073 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69192234-00f9-4d19-b9a8-0fb69fb25963 service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:49.002682 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47284e32-889e-4f69-9f6b-221bd5023a93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:49.003297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47284e32-889e-4f69-9f6b-221bd5023a93 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:49.003673 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47284e32-889e-4f69-9f6b-221bd5023a93 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:49.004163 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47284e32-889e-4f69-9f6b-221bd5023a93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:49.006058 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-47284e32-889e-4f69-9f6b-221bd5023a93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:49.019522 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47284e32-889e-4f69-9f6b-221bd5023a93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:49.021164 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47284e32-889e-4f69-9f6b-221bd5023a93 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:49.032400 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1437/2877] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:42:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:42:58.334463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=29 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:42:58.943979 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5618a3-b6db-49d4-957d-17f8961e2ef5 service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:42:58.945486 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-bb5618a3-b6db-49d4-957d-17f8961e2ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:58.964139 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5618a3-b6db-49d4-957d-17f8961e2ef5 service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:42:58.970788 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99a271-c795-433e-8260-9d96091c76dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:42:58.971393 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99a271-c795-433e-8260-9d96091c76dc service nova] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:42:58.972111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99a271-c795-433e-8260-9d96091c76dc service nova] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:42:58.972111 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99a271-c795-433e-8260-9d96091c76dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:42:58.973944 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2a99a271-c795-433e-8260-9d96091c76dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:42:58.987011 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99a271-c795-433e-8260-9d96091c76dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:42:58.988929 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99a271-c795-433e-8260-9d96091c76dc service nova] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:42:59.001470 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1441/2878] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:42:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:42:59.027328 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=66 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:43:01.680709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47284e32-889e-4f69-9f6b-221bd5023a93 service nova] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:43:01.691107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47284e32-889e-4f69-9f6b-221bd5023a93 service nova] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:43:01.694679 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d842f7f-939d-4f54-bbdb-6d45f94fb6d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:43:01.695271 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d842f7f-939d-4f54-bbdb-6d45f94fb6d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:43:01.695549 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d842f7f-939d-4f54-bbdb-6d45f94fb6d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:43:01.695823 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d842f7f-939d-4f54-bbdb-6d45f94fb6d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:43:01.697571 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9d842f7f-939d-4f54-bbdb-6d45f94fb6d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:43:01.761264 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d842f7f-939d-4f54-bbdb-6d45f94fb6d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:43:01.762709 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d842f7f-939d-4f54-bbdb-6d45f94fb6d2 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:43:01.772580 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1438/2879] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:43:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:43:08.996584 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=6 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:43:11.768247 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=30 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:44:15.914659 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99a271-c795-433e-8260-9d96091c76dc service nova] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:44:15.916096 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-2a99a271-c795-433e-8260-9d96091c76dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:44:15.928428 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99a271-c795-433e-8260-9d96091c76dc service nova] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:44:15.933300 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51600f-b073-459f-b309-6f43a4ae0be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:44:15.933939 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51600f-b073-459f-b309-6f43a4ae0be9 service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:15.934453 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51600f-b073-459f-b309-6f43a4ae0be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:15.934764 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51600f-b073-459f-b309-6f43a4ae0be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:15.937410 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-fa51600f-b073-459f-b309-6f43a4ae0be9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:44:15.949042 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51600f-b073-459f-b309-6f43a4ae0be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:44:15.950411 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51600f-b073-459f-b309-6f43a4ae0be9 service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:44:15.962438 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1442/2880] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:44:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:44:16.099819 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22763a-401f-4907-aed7-fe99da330d7e None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:16.100443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22763a-401f-4907-aed7-fe99da330d7e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:16.100647 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22763a-401f-4907-aed7-fe99da330d7e None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:16.102112 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1439/2881] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:44:16.108117 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26094b-4bab-4bc1-bc4e-7e4ae38beb10 None None] REQUEST_METHOD: `POST` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:16.108404 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26094b-4bab-4bc1-bc4e-7e4ae38beb10 None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:16.108676 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26094b-4bab-4bc1-bc4e-7e4ae38beb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:16.152180 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.auth.core [None req-4b26094b-4bab-4bc1-bc4e-7e4ae38beb10 None None] MFA Rules not processed for user `9e0b7efc488a42ffab8ba7e2d101ced4`. Rule list: `[]` (Enabled: `True`). {{(pid=81550) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:44:16.153778 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-4b26094b-4bab-4bc1-bc4e-7e4ae38beb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:44:16.161930 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1443/2882] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:44:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:44:16.175102 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22763a-401f-4907-aed7-fe99da330d7e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:44:16.187117 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22763a-401f-4907-aed7-fe99da330d7e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:44:16.190941 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcde66-df3d-41df-9acb-0b44997df904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:44:16.191606 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcde66-df3d-41df-9acb-0b44997df904 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:16.191894 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcde66-df3d-41df-9acb-0b44997df904 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:16.192160 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcde66-df3d-41df-9acb-0b44997df904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:16.201194 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcde66-df3d-41df-9acb-0b44997df904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:44:16.202567 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcde66-df3d-41df-9acb-0b44997df904 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:44:16.210931 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1440/2883] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:44:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:44:16.307517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156440a-aba0-4283-872d-7cd24794aadc None None] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:16.307830 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156440a-aba0-4283-872d-7cd24794aadc None None] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:16.308067 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156440a-aba0-4283-872d-7cd24794aadc None None] PATH_INFO: `/` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:16.309742 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1444/2884] 10.222.0.55 () {54 vars in 921 bytes} [Fri Aug 19 16:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 16:44:16.314628 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c8c1d-cd2d-48a6-89a4-c1318679c95c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:16.315052 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c8c1d-cd2d-48a6-89a4-c1318679c95c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:16.315434 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c8c1d-cd2d-48a6-89a4-c1318679c95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:16.369466 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b12c8c1d-cd2d-48a6-89a4-c1318679c95c None None] MFA Rules not processed for user `4739e80619a54b178b9aacec6a42a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 16:44:16.371421 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b12c8c1d-cd2d-48a6-89a4-c1318679c95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:44:16.383726 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1441/2885] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Aug 19 16:44:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 16:44:16.395037 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156440a-aba0-4283-872d-7cd24794aadc None None] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:44:16.406963 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156440a-aba0-4283-872d-7cd24794aadc None None] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:44:16.410708 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7f13e-3819-4800-b56c-aba6a595fc7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:44:16.411325 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7f13e-3819-4800-b56c-aba6a595fc7d service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:16.411603 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7f13e-3819-4800-b56c-aba6a595fc7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:16.411884 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7f13e-3819-4800-b56c-aba6a595fc7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:16.424964 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7f13e-3819-4800-b56c-aba6a595fc7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:44:16.426596 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7f13e-3819-4800-b56c-aba6a595fc7d service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:44:16.443519 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1445/2886] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:44:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:44:26.364607 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=15 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:44:26.435937 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=21 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:44:34.008524 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c8c1d-cd2d-48a6-89a4-c1318679c95c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:44:34.021324 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c8c1d-cd2d-48a6-89a4-c1318679c95c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:44:34.027458 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8b9c2-4d4e-4085-ae4f-e6831c6156fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4739e80619a54b178b9aacec6a42a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:44:34.028070 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8b9c2-4d4e-4085-ae4f-e6831c6156fe service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:34.028349 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8b9c2-4d4e-4085-ae4f-e6831c6156fe service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:34.028644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8b9c2-4d4e-4085-ae4f-e6831c6156fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:34.031469 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3bb8b9c2-4d4e-4085-ae4f-e6831c6156fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:44:34.043980 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8b9c2-4d4e-4085-ae4f-e6831c6156fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:44:34.045606 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8b9c2-4d4e-4085-ae4f-e6831c6156fe service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:44:34.057905 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1442/2887] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:44:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:44:44.053232 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=6 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:44:46.465275 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7f13e-3819-4800-b56c-aba6a595fc7d service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:44:46.474002 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7f13e-3819-4800-b56c-aba6a595fc7d service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:44:46.477615 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2750d-e64f-442d-a380-1993981e7dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0b7efc488a42ffab8ba7e2d101ced4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:44:46.478253 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2750d-e64f-442d-a380-1993981e7dbf service neutron] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:46.478492 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2750d-e64f-442d-a380-1993981e7dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:46.478745 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2750d-e64f-442d-a380-1993981e7dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:46.480809 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-cad2750d-e64f-442d-a380-1993981e7dbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:44:46.542593 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2750d-e64f-442d-a380-1993981e7dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:44:46.544022 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2750d-e64f-442d-a380-1993981e7dbf service neutron] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:44:46.553583 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1446/2888] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:44:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:44:52.502957 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8b9c2-4d4e-4085-ae4f-e6831c6156fe service nova] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:44:52.504283 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3bb8b9c2-4d4e-4085-ae4f-e6831c6156fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:44:52.520459 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8b9c2-4d4e-4085-ae4f-e6831c6156fe service nova] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:44:52.525559 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfb521-c397-462f-8911-ae339baf905f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:44:52.526172 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfb521-c397-462f-8911-ae339baf905f service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:52.526463 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfb521-c397-462f-8911-ae339baf905f service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:52.526749 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfb521-c397-462f-8911-ae339baf905f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:52.529360 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2ccfb521-c397-462f-8911-ae339baf905f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:44:52.544153 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfb521-c397-462f-8911-ae339baf905f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:44:52.545022 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfb521-c397-462f-8911-ae339baf905f service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:44:52.556644 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1443/2889] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:44:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:44:52.570810 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2750d-e64f-442d-a380-1993981e7dbf service neutron] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:44:52.572317 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-cad2750d-e64f-442d-a380-1993981e7dbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:44:52.584920 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2750d-e64f-442d-a380-1993981e7dbf service neutron] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:44:52.590234 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:44:52.590874 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:52.591222 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:52.591418 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] PATH_INFO: `/v3/users/f989b6fded53458b85b39789a32a39e4` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:52.592562 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f989b6fded53458b85b39789a32a39e4)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:44:52.594514 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:44:52.644356 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f989b6fded53458b85b39789a32a39e4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:52.645125 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f989b6fded53458b85b39789a32a39e4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:52.645728 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f989b6fded53458b85b39789a32a39e4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:52.653031 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f989b6fded53458b85b39789a32a39e4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:52.664906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f989b6fded53458b85b39789a32a39e4'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:52.678136 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1447/2890] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:44:52 2022] DELETE /identity/v3/users/f989b6fded53458b85b39789a32a39e4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:44:52.853695 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfb521-c397-462f-8911-ae339baf905f service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:44:52.867161 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfb521-c397-462f-8911-ae339baf905f service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:44:52.925727 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:44:52.926367 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:52.926649 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:52.926951 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] PATH_INFO: `/v3/projects/5181e8549b3a439c8921319d6957f670` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:52.928086 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5181e8549b3a439c8921319d6957f670)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:44:52.930038 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:44:52.976184 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5181e8549b3a439c8921319d6957f670', 'request_id': 'req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:52.982265 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5181e8549b3a439c8921319d6957f670', 'request_id': 'req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:52.990096 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5181e8549b3a439c8921319d6957f670', 'request_id': 'req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.003033 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] The token cache is being invalidate because project 5181e8549b3a439c8921319d6957f670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:44:53.003320 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.003946 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.005433 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1444/2891] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:44:52 2022] DELETE /identity/v3/projects/5181e8549b3a439c8921319d6957f670 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:44:53.009517 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:44:53.010921 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:44:53.023988 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab3691-8d59-49cd-8c5e-bdc5b9d730c0 admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:44:53.080463 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:44:53.081054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:53.081329 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:53.081619 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] PATH_INFO: `/v3/users/940ab95bcc124e7fb037fccf0b95fdf9` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:53.082726 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] RBAC: Authorizing `identity:delete_user(user_id=940ab95bcc124e7fb037fccf0b95fdf9)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:44:53.084692 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:44:53.090677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:44:53.135424 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '940ab95bcc124e7fb037fccf0b95fdf9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.136071 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '940ab95bcc124e7fb037fccf0b95fdf9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.136682 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '940ab95bcc124e7fb037fccf0b95fdf9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.143644 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '940ab95bcc124e7fb037fccf0b95fdf9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.157734 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '940ab95bcc124e7fb037fccf0b95fdf9'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.171934 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1448/2892] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:44:53 2022] DELETE /identity/v3/users/940ab95bcc124e7fb037fccf0b95fdf9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:44:53.342828 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:44:53.344272 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:44:53.358582 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21fdcd3-01e0-4f6e-aab7-ae0f312a2f33 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:44:53.414655 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:44:53.415291 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:44:53.415572 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:44:53.415854 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] PATH_INFO: `/v3/projects/1387bc55d4a249638617853ef4b12046` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:44:53.416995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1387bc55d4a249638617853ef4b12046)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:44:53.419843 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:44:53.440512 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:44:53.468011 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1387bc55d4a249638617853ef4b12046', 'request_id': 'req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.473676 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1387bc55d4a249638617853ef4b12046', 'request_id': 'req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.479743 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1387bc55d4a249638617853ef4b12046', 'request_id': 'req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.492948 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] The token cache is being invalidate because project 1387bc55d4a249638617853ef4b12046 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:44:53.493243 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.493864 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:44:53.495325 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1445/2893] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:44:53 2022] DELETE /identity/v3/projects/1387bc55d4a249638617853ef4b12046 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:45:03.167261 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:45:03.489949 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:45:51.532867 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:45:51.534315 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:45:51.604492 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beacb264-7b2f-41c6-9f52-af974bbc816f admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:45:51.666868 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d611885-c6d5-4ae0-bd28-94937a0baf65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:45:51.667501 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d611885-c6d5-4ae0-bd28-94937a0baf65 service placement] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:45:51.667794 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d611885-c6d5-4ae0-bd28-94937a0baf65 service placement] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:45:51.668137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d611885-c6d5-4ae0-bd28-94937a0baf65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:45:51.670137 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-4d611885-c6d5-4ae0-bd28-94937a0baf65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:45:51.682899 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d611885-c6d5-4ae0-bd28-94937a0baf65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:45:51.684472 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d611885-c6d5-4ae0-bd28-94937a0baf65 service placement] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:45:51.695295 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1449/2894] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:45:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:45:55.481046 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:45:55.483099 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:45:55.500044 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd73e31-95fc-4faa-bd6f-9bdf0270aab3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:45:55.505412 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c196ab4-00e6-406c-a44d-65df0843c7c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:45:55.506021 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c196ab4-00e6-406c-a44d-65df0843c7c8 service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:45:55.506297 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c196ab4-00e6-406c-a44d-65df0843c7c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:45:55.507426 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c196ab4-00e6-406c-a44d-65df0843c7c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:45:55.513734 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9c196ab4-00e6-406c-a44d-65df0843c7c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:45:55.537638 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c196ab4-00e6-406c-a44d-65df0843c7c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:45:55.539428 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c196ab4-00e6-406c-a44d-65df0843c7c8 service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:45:55.550809 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1446/2895] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:45:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 16:45:58.420819 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d611885-c6d5-4ae0-bd28-94937a0baf65 service placement] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:45:58.425886 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-4d611885-c6d5-4ae0-bd28-94937a0baf65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:45:58.495675 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d611885-c6d5-4ae0-bd28-94937a0baf65 service placement] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:45:58.500211 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:45:58.500867 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:45:58.501148 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:45:58.501420 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] PATH_INFO: `/v3/users/886a581760e547b6a15856da84ef3859` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:45:58.502534 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] RBAC: Authorizing `identity:delete_user(user_id=886a581760e547b6a15856da84ef3859)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:45:58.504445 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:45:58.542761 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:45:58.552827 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '886a581760e547b6a15856da84ef3859'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:58.553467 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '886a581760e547b6a15856da84ef3859'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:58.554076 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '886a581760e547b6a15856da84ef3859'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:58.561241 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '886a581760e547b6a15856da84ef3859'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:58.573952 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '886a581760e547b6a15856da84ef3859'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:58.587293 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1450/2896] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:45:58 2022] DELETE /identity/v3/users/886a581760e547b6a15856da84ef3859 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:45:58.756708 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c196ab4-00e6-406c-a44d-65df0843c7c8 service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:45:58.767697 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c196ab4-00e6-406c-a44d-65df0843c7c8 service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:45:58.823359 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:45:58.823986 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:45:58.824259 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:45:58.824531 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] PATH_INFO: `/v3/projects/363667c09d4e481da5767f1043d57e76` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:45:58.825652 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] RBAC: Authorizing `identity:delete_project(project_id=363667c09d4e481da5767f1043d57e76)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:45:58.827529 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:45:58.874496 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '363667c09d4e481da5767f1043d57e76', 'request_id': 'req-b2445651-9c15-44b9-8f27-0b504a66aabf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:58.879875 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '363667c09d4e481da5767f1043d57e76', 'request_id': 'req-b2445651-9c15-44b9-8f27-0b504a66aabf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:58.886403 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '363667c09d4e481da5767f1043d57e76', 'request_id': 'req-b2445651-9c15-44b9-8f27-0b504a66aabf'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:58.899069 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] The token cache is being invalidate because project 363667c09d4e481da5767f1043d57e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:45:58.899356 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:58.899996 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:58.901485 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1447/2897] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:45:58 2022] DELETE /identity/v3/projects/363667c09d4e481da5767f1043d57e76 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:45:58.905488 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:45:58.906906 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:45:58.920220 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f9a70-3ded-4e66-ada7-772ebed81cde admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:45:58.978054 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:45:58.978677 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:45:58.978971 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:45:58.979218 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] PATH_INFO: `/v3/users/6e778c57ba4949ec8d214aa138740b84` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:45:58.980362 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e778c57ba4949ec8d214aa138740b84)` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:45:58.982297 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:45:59.028816 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e778c57ba4949ec8d214aa138740b84'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:59.029493 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e778c57ba4949ec8d214aa138740b84'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:59.030229 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e778c57ba4949ec8d214aa138740b84'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:59.037732 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e778c57ba4949ec8d214aa138740b84'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:59.049468 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.notifications [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e778c57ba4949ec8d214aa138740b84'} {{(pid=81550) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:59.062124 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1451/2898] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Aug 19 16:45:58 2022] DELETE /identity/v3/users/6e778c57ba4949ec8d214aa138740b84 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 16:45:59.232065 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:45:59.233558 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:45:59.248277 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2445651-9c15-44b9-8f27-0b504a66aabf admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:45:59.306724 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3904bee-660c-4ba3-b19f-1a0505f0c69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1decd3055af14f42987a890baebb81a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d450c451d94f27b564ec08cc487161', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:45:59.307353 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3904bee-660c-4ba3-b19f-1a0505f0c69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:45:59.307628 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3904bee-660c-4ba3-b19f-1a0505f0c69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:45:59.307900 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3904bee-660c-4ba3-b19f-1a0505f0c69c admin admin] PATH_INFO: `/v3/projects/85e3f45084f140fd9cd6dbcbb1f0eaa2` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:45:59.309024 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3904bee-660c-4ba3-b19f-1a0505f0c69c admin admin] RBAC: Authorizing `identity:delete_project(project_id=85e3f45084f140fd9cd6dbcbb1f0eaa2)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:45:59.310917 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3904bee-660c-4ba3-b19f-1a0505f0c69c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:45:59.332125 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:45:59.359553 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f3904bee-660c-4ba3-b19f-1a0505f0c69c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85e3f45084f140fd9cd6dbcbb1f0eaa2', 'request_id': 'req-f3904bee-660c-4ba3-b19f-1a0505f0c69c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:59.365851 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f3904bee-660c-4ba3-b19f-1a0505f0c69c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85e3f45084f140fd9cd6dbcbb1f0eaa2', 'request_id': 'req-f3904bee-660c-4ba3-b19f-1a0505f0c69c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:59.372443 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f3904bee-660c-4ba3-b19f-1a0505f0c69c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85e3f45084f140fd9cd6dbcbb1f0eaa2', 'request_id': 'req-f3904bee-660c-4ba3-b19f-1a0505f0c69c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:59.384536 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f3904bee-660c-4ba3-b19f-1a0505f0c69c admin admin] The token cache is being invalidate because project 85e3f45084f140fd9cd6dbcbb1f0eaa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 16:45:59.384866 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f3904bee-660c-4ba3-b19f-1a0505f0c69c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:59.385487 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f3904bee-660c-4ba3-b19f-1a0505f0c69c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 16:45:59.386995 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1448/2899] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Aug 19 16:45:59 2022] DELETE /identity/v3/projects/85e3f45084f140fd9cd6dbcbb1f0eaa2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 16:46:09.058681 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG dbcounter [-] [81550] Writing DB stats keystone:SELECT=69,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81550) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:46:09.382107 dv-cinder-831247-23-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 16:51:15.443481 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] Authenticating user token {{(pid=81550) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 16:51:15.444985 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:51:15.580857 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee3265-0625-4b4b-8eaf-2d23b107f7ff admin admin] Validating token access rules against request {{(pid=81550) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 16:51:15.648627 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a4b95-c7bb-4270-9e80-a8049a669506 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dcbe2b128a94b9abbf3a9b25db1e845', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd143cf444f354ec0bf2089dae3d94255', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81550) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 16:51:15.649278 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a4b95-c7bb-4270-9e80-a8049a669506 service placement] REQUEST_METHOD: `GET` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 16:51:15.649557 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a4b95-c7bb-4270-9e80-a8049a669506 service placement] SCRIPT_NAME: `/identity` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 16:51:15.649829 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a4b95-c7bb-4270-9e80-a8049a669506 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81550) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 16:51:15.651617 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.fernet_utils [None req-c43a4b95-c7bb-4270-9e80-a8049a669506 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81550) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 16:51:15.664409 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a4b95-c7bb-4270-9e80-a8049a669506 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 16:51:15.666027 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a4b95-c7bb-4270-9e80-a8049a669506 service placement] RBAC: Authorization granted {{(pid=81550) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 16:51:15.676913 dv-cinder-831247-23-iscsi devstack@keystone.service[81550]: [pid: 81550|app: 0|req: 1452/2900] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Aug 19 16:51:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)